AI Glossary

Pass@k

Also called pass at k

What does “Pass@k” mean?

Pass@k: The probability that at least one of k attempts solves a task, commonly estimated across many benchmark problems. It measures what a system can achieve with several tries.

Pass@k visual guide: The probability that at least one of k attempts solves a task, commonly estimated across many benchmark problems. It measures what a system can achieve with several tries.
Visual guide Open full size (new tab)

What is an example of “Pass@k”?

A code benchmark reports pass@1 for one attempt and pass@10 when ten generated answers may be tested.

Keep learning

See also

BenchmarkTermSWE-benchTermEvalsTerm

Same topic

PerplexityTermOut-of-distribution detectionTermPersonal information redactionTermModel poisoningTermPrecisionTerm