AI Glossary
Benchmark
What does “Benchmark” mean?
Benchmark: Fixed, shared set of tasks with a scoring method, used to compare AI models on the same test. Benchmarks cover skills like reasoning, coding, maths, knowledge and safety. They make progress measurable, but scores can overstate real-world ability if a test is narrow, saturated or leaked into training data.
What is an example of “Benchmark”?
A coding benchmark might ask models to fix real bugs from open-source projects and count how many fixes pass the project's tests.