AI Glossary

Reward model

What does “Reward model” mean?

Reward model: A separate model trained to predict how good a response is, usually from human or AI preference data. During reinforcement learning it stands in for human judges and scores the main model's outputs at a scale people could not manage.

Reward model visual guide: A separate model trained to predict how good a response is, usually from human or AI preference data. During reinforcement learning it stands in for human judges and scores the main model's outputs at a scale people could not manage.
Visual guide Open full size (new tab)

What is an example of “Reward model”?

If a reward model learns to favor long answers, the trained model may become needlessly wordy. This is a simple case of reward hacking.

Keep learning

See also

Reinforcement learning from human feedbackTermReinforcement learning from AI feedbackTermDirect preference optimizationTermReward hackingTerm

Same topic

Reinforcement learningTermReinforcement learning from verifiable rewardsTermScaling lawsTermSynthetic dataTermTransfer learningTerm