AI Glossary

Gradient clipping

What does “Gradient clipping” mean?

Gradient clipping: Limiting a gradient's size before updating parameters.

Gradient clipping visual guide: Limiting a gradient's size before updating parameters.
Visual guide Open full size (new tab)

What is an example of “Gradient clipping”?

A trainer clips an unusually large gradient to avoid an unstable jump.

Keep learning

See also

TrainingTermRegularizationTermLearning rateTermEarly stoppingTermLearning rate scheduleTerm

Same topic

Gradient accumulationTermGradient descentTermHard negativeTermFine-tuningTermHyperparameterTerm