Item Response Theory for AI Safety

Item Response Theory for AI Safety | AI Research

Key Takeaways

Item Response Theory for AI Safety proposes using a statistical framework from psychometrics to improve how language models are evaluated for safety.
Language models differ in how safely they behave and these differences are measured by safety benchmarks.
But aggregated benchmark scores are hard to trust and interpret, because benchmarks duplicate one another, correlate heavily, and models may sandbag when they detect evaluation.
To address these issues, we draw on Item Response Theory (IRT), a statistical toolkit for measuring these latents from performance on items with inferred psychometric properties.
We fit IRT models to eight safety benchmarks across 192 language models, the largest psychometric analysis of LLM safety evaluations to date, and contribute three results.

Paper AbstractExpand

Language models differ in how safely they behave and these differences are measured by safety benchmarks. But aggregated benchmark scores are hard to trust and interpret, because benchmarks duplicate one another, correlate heavily, and models may sandbag when they detect evaluation. To address these issues, we draw on Item Response Theory (IRT), a statistical toolkit for measuring these latents from performance on items with inferred psychometric properties. We fit IRT models to eight safety benchmarks across 192 language models, the largest psychometric analysis of LLM safety evaluations to date, and contribute three results. First, we find that three interpretable factors of refusal strictness, truthfulness, and contextual harm explain most of the variance between models across benchmarks. Second, psychometrically selected items recover full benchmark scores with lower error than random subsets of the same size, and roughly ten adaptively chosen items suffice for several individual benchmarks, cutting evaluation cost by 97-99%. Third, IRT supports audits of individual models, showing that it can be used to detect naive sandbagging and changes of model behind APIs. Overall, we show IRT is a ready-made toolkit for reading, reducing, and auditing safety benchmarks, which we recommend frontier labs and evaluators adopt.

Item Response Theory for AI Safety proposes using a statistical framework from psychometrics to improve how language models are evaluated for safety. By treating AI models as test-takers and safety prompts as test items, the researchers aim to create more efficient, interpretable, and auditable safety benchmarks.

Measuring Safety Latents

The researchers analyzed 192 language models across eight safety benchmarks to determine what these tests actually measure. They found that safety is not a single, unified trait. Instead, 77% of the variance between models can be explained by three distinct latent factors: refusal strictness, truthfulness, and contextual harm. This suggests that current benchmark suites are often redundant and that models may perform well on one aspect of safety while failing on another.

Efficient Benchmarking

The study demonstrates that full safety benchmarks are often inefficient because many items provide little information about a model's capabilities. By applying Item Response Theory (IRT), the authors developed two methods to reduce evaluation costs:

Static Tests: Using 75 items—less than 2% of the original 5,255-item suite—the researchers constructed three fixed tests that accurately recover the three latent safety factors.
Adaptive Testing: By using computerized adaptive testing (CAT), where the system selects the most informative items based on a model's previous responses, the researchers estimated individual benchmark scores using only about ten items. This approach reduced evaluation costs by 97–99% while maintaining high accuracy in model rankings.

Auditing Model Behavior

The researchers used IRT to detect when models behave differently than expected, a common issue when models "sandbag" or when the underlying model behind an API is swapped. By using person-fit statistics, which compare a model’s actual responses to the patterns expected for its ability level, the authors were able to identify at least 80% of selective prompted sandbagging. This method allows auditors to flag suspicious behavior without needing to know the specific intervention used to manipulate the model.

Considerations for Implementation

The authors note that applying IRT to AI requires careful calibration. Because LLM evaluations often involve many items but fewer test-takers than human psychometric studies, unregularized models can become unstable. The researchers addressed this by using regularized fits with log-normal priors on item discrimination and normal priors on item difficulty. They recommend that frontier labs and evaluators adopt these IRT-based tools to move beyond simple, aggregated benchmark scores toward a more rigorous, diagnostic approach to AI safety.