AI Glossary

Weight decay

What does “Weight decay” mean?

Weight decay: A technique that nudges parameters toward smaller values during training.

Weight decay visual guide: A technique that nudges parameters toward smaller values during training.
Visual guide Open full size (new tab)

What is an example of “Weight decay”?

A trainer applies weight decay to discourage unusually large weights.

Keep learning

See also

TrainingTermRegularizationTermLearning rateTermAdam optimizerTermMomentum in optimizationTerm

Same topic

DropoutTermEarly stoppingTermWeight initializationTermWeak supervisionTermValue functionTerm