AI GitHub
Outlines for typed model outputs
Generate text that conforms to a type, schema or pattern through a supported model backend.
Key Takeaways
- Outlines represents output constraints in Python and connects them to supported generation backends.
- Pick the output constraint and the inference backend together.
- Turn one text-only extraction prompt into a typed object with an optional field.
- A valid object can still contain a wrong value.
How it works
Outlines represents output constraints in Python and connects them to supported generation backends. With constrained decoding, the runtime rules out continuations that would break the requested format.
What you need
Pick the output constraint and the inference backend together. A local decoder integration can expose different control from an application that only calls a hosted endpoint.
What to try first
Turn one text-only extraction prompt into a typed object with an optional field. Feed it an incomplete document and check how it represents missing information.
Include a source sentence containing quotes and backslashes. A useful regression fixture checks both the generated value and its representation once the output has been parsed.
Where to look for failures
A valid object can still contain a wrong value. Add checks against the original text and make the schema expressive enough for ambiguity and refusal.
Related Franklin pages
Use a constraint that permits several valid outputs. That makes it easier to inspect how each programming model represents choices and where it enforces them.
- Interleave literal text, generated spans and output constraints in a program. Guidance for composing prompts and constrained generation
- Experiment with a query language that puts model calls and constraints in the same program. LMQL for studying constrained LLM programs
For the distinction between requesting structured output and enforcing its format, see Franklin’s glossary. Structured output
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗