AI Glossary

Regularization

What does “Regularization” mean?

Regularization: Techniques that discourage a model from fitting its training data too closely, so it generalizes better to new examples. Common methods include weight decay, which penalizes large weights; dropout, which randomly switches off parts of the network during training; and stopping training early.

Regularization visual guide: Techniques that discourage a model from fitting its training data too closely, so it generalizes better to new examples. Common methods include weight decay, which penalizes large weights; dropout, which randomly switches off parts of the network during training; and stopping training early.
Visual guide Open full size (new tab)

What is an example of “Regularization”?

If validation scores start falling while training scores keep improving, adding regularization or stopping earlier is a standard fix.

Keep learning

See also

OverfittingTermUnderfittingTermValidation and test setsTermLoss functionTerm

Same topic

Reinforcement learning from AI feedbackTermQLoRATermReinforcement learning from human feedbackTermPre-trainingTermReinforcement learning from verifiable rewardsTerm