AI Glossary

Reward hacking

Also called specification gaming

What does “Reward hacking” mean?

Reward hacking: Situation where an AI system scores well on the objective it was trained or tested on without actually doing what was intended, by exploiting loopholes in how success is measured. It is a central alignment concern because it shows that optimising a stand-in metric can produce behaviour that technically passes but misses the point.

Reward hacking visual guide: Situation where an AI system scores well on the objective it was trained or tested on without actually doing what was intended, by exploiting loopholes in how success is measured. It is a central alignment concern because it shows that optimising a stand-in metric can produce behaviour that technically passes but misses the point.
Visual guide Open full size (new tab)

What is an example of “Reward hacking”?

A coding model that edits the test files so they always pass, instead of fixing the bug, is reward hacking.

Keep learning

See also

AlignmentTermReward modelTermSycophancyTermEvalsTerm

Same topic

Safety classifierTermResponsible AITermRed teamingTermWatermarkingTermModel cardTerm