AI GitHub
Instructor for validating model responses in Python
Parse model responses into typed Python objects and retry when validation fails.
Key Takeaways
- Instructor wraps provider clients and uses Pydantic models to express the desired fields and validation rules.
- Bring a provider client and a Pydantic output model.
- Extract one object with an enum and a validator for a value that must appear in the input.
- A valid typed response can still misstate a fact, so use deterministic checks where the source makes verification possible.
How it works
Instructor wraps provider clients and uses Pydantic models to express the desired fields and validation rules. Failed validation can be sent back into another attempt, depending on client configuration.
What you need
Bring a provider client and a Pydantic output model. Choose a strict validator for facts you can check independently, such as a known identifier or an allowed range.
What to try first
Extract one object with an enum and a validator for a value that must appear in the input. Count retry attempts as well as final successes when testing ambiguous examples.
Log validation errors and request counts for each input. A tool that produces excellent final objects at the cost of frequent retries can be expensive on noisy records.
Where to look for failures
Retries add latency and provider calls. A valid typed response can still misstate a fact, so use deterministic checks where the source makes verification possible.
Related Franklin pages
Test both a valid structured response and one that requires recovery. A good application contract specifies what happens after the allowed attempts are exhausted.
- Bring schema-based parsing and retries to JavaScript model clients. Instructor JS for schema-validated TypeScript responses
- Build Python agents with typed dependencies, tools and model outputs. Pydantic AI for typed agent applications
For the distinction between requesting structured output and enforcing its format, see Franklin’s glossary. Structured output
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗