Guidance: Constrained LLM Generation
When to Use This Skill
Use Guidance when you need to:
- Control LLM output syntax with regex or grammars
- Guarantee valid JSON/XML/code generation
- Reduce latency vs traditional prompting approaches
- Enforce structured formats (dates, emails, IDs, etc.)
- Build multi-step workflows with Pythonic control flow
- Prevent invalid outputs through grammatical constraints
GitHub Stars: 18,000+ | From: Microsoft Research
Installation
pip install guidance
pip install guidance[transformers]
pip install guidance[llama_cpp]
Quick Start
Basic Example: Structured Generation
from guidance import models, gen
lm = models.OpenAI("gpt-4")
result = lm + "The capital of France is " + gen("capital", max_tokens=5)
print(result["capital"])
With Anthropic Claude
from guidance import models, gen, system, user, assistant
lm = models.Anthropic("claude-sonnet-4-5-20250929")
with system():
lm += "You are a helpful assistant."
with user():
lm += "What is the capital of France?"
with assistant():
lm += gen(max_tokens=20)
Core Concepts
1. Context Managers
Guidance uses Pythonic context managers for chat-style interactions.
from guidance import system, user, assistant, gen
lm = models.Anthropic("claude-sonnet-4-5-20250929")
with system():
lm += "You are a JSON generation expert."
with user():
lm += "Generate a person object with name and age."
with assistant():
lm += gen("response", max_tokens=100)
print(lm["response"])
Benefits:
- Natural chat flow
- Clear role separation
- Easy to read and maintain
2. Constrained Generation
Guidance ensures outputs match specified patterns using regex or grammars.
Regex Constraints
from guidance import models, gen
lm = models.Anthropic("claude-sonnet-4-5-20250929")
lm += "Email: " + gen("email", regex=r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")
lm += "Date: " + gen("date", regex=r"\d{4}-\d{2}-\d{2}")
lm += "Phone: " + gen("phone", regex=r"\d{3}-\d{3}-\d{4}")
print(lm["email"])
print(lm["date"])
How it works:
- Regex converted to grammar at token level
- Invalid tokens filtered during generation
- Model can only produce matching outputs
Selection Constraints
from guidance import models, gen, select
lm = models.Anthropic("claude-sonnet-4-5-20250929")
lm += "Sentiment: " + select(["positive", "negative", "neutral"], name="sentiment")
lm += "Best answer: " + select(
["A) Paris", "B) London", "C) Berlin", "D) Madrid"],
name="answer"
)
print(lm["sentiment"])
print(lm["answer"])
3. Token Healing
Guidance automatically "heals" token boundaries between prompt and generation.
Problem: Tokenization creates unnatural boundaries.
prompt = "The capital of France is "
Solution: Guidance backs up one token and regenerates.
from guidance import models, gen
lm = models.Anthropic("claude-sonnet-4-5-20250929")
lm += "The capital of France is " + gen("capital", max_tokens=5)
Benefits:
- Natural text boundaries
- No awkward spacing issues
- Better model performance (sees natural token sequences)
4. Grammar-Based Generation
Define complex structures using context-free grammars.
from guidance import models, gen
lm = models.Anthropic("claude-sonnet-4-5-20250929")
json_grammar = """
{
"name": <gen name regex="[A-Za-z ]+" max_tokens=20>,
"age": <gen age regex="[0-9]+" max_tokens=3>,
"email": <gen email regex="[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}" max_tokens=50>
}
"""
lm += gen("person", grammar=json_grammar)
print(lm["person"])
Use cases:
- Complex structured outputs
- Nested data structures
- Programming language syntax
- Domain-specific languages
5. Guidance Functions
Create reusable generation patterns with the @guidance decorator.
from guidance import guidance, gen, models
@guidance
def generate_person(lm):
"""Generate a person with name and age."""
lm += "Name: " + gen("name", max_tokens=20, stop="\n")
lm += "\nAge: " + gen("age", regex=r"[0-9]+", max_tokens=3)
return lm
lm = models.Anthropic("claude-sonnet-4-5-20250929")
lm = generate_person(lm)
print(lm["name"])
print(lm["age"])
Stateful Functions:
@guidance(stateless=False)
def react_agent(lm, question, tools, max_rounds=5):
"""ReAct agent with tool use."""
lm += f"Question:
