AI Glossary

Evals

Also called evaluations

What does “Evals” mean?

Evals: Structured tests used to measure what an AI model can do, how reliably it does it, and whether it behaves safely. Evals can be public benchmarks, private test suites built for one product, human ratings, or automated grading. Teams run them before releases and after changes to catch regressions.

Evals visual guide: Structured tests used to measure what an AI model can do, how reliably it does it, and whether it behaves safely. Evals can be public benchmarks, private test suites built for one product, human ratings, or automated grading. Teams run them before releases and after changes to catch regressions.
Visual guide Open full size (new tab)

What is an example of “Evals”?

A company building a legal assistant might maintain 500 real contract questions with expert answers and re-run them on every model update.

Keep learning

See also

BenchmarkTermLLM-as-a-judgeTermRed teamingTermAgent evalsTerm

More from Franklin

On Franklin

Benchmark methodologyPage

Same topic

Arena rankingsTermFrontier safety frameworkTermLeaderboardTermMMLUTermModel cardTerm