Guidance for composing prompts and constrained generation

How it works

A Guidance program can construct a prompt while naming the places where a model may generate text. Choices and grammars give those spans boundaries that ordinary string prompts leave implicit.

What you need

Start with a backend and a short program you can inspect in full. Prompt construction and output capture both affect how the calling code uses generated spans.

What to try first

Build a classification prompt that combines a fixed prefix with a short choice list. Use uncertain examples to see whether an explicit fallback choice is needed.

Put two generated variables into the same template and check whether one constrains the other. Inspect the rendered prompt to see what text the model actually receives.

Where to look for failures

A fixed choice set can hide genuine uncertainty. Support for grammar constraints depends on the model backend, so check before moving an example to a different endpoint.

Related Franklin pages

Use a constraint that permits several valid outputs. That makes it easier to inspect how each programming model represents choices and where it enforces them.

• Experiment with a query language that puts model calls and constraints in the same program. LMQL for studying constrained LLM programs
• Generate text that conforms to a type, schema or pattern through a supported model backend. Outlines for typed model outputs

For the distinction between requesting structured output and enforcing its format, see Franklin’s glossary. Structured output

Project documentation

• Current project README
• README at the reviewed revision