AI Glossary

Stochastic gradient descent

Also called SGD

What does “Stochastic gradient descent” mean?

Stochastic gradient descent: Gradient descent that estimates each update from a small sample or mini-batch.

Stochastic gradient descent visual guide: Gradient descent that estimates each update from a small sample or mini-batch.
Visual guide Open full size (new tab)

What is an example of “Stochastic gradient descent”?

A model updates after each batch rather than waiting for every training record.

Keep learning

See also

TrainingTermRegularizationTermLearning rateTermOptimizerTermAdam optimizerTerm

Same topic

Momentum in optimizationTermSynthetic dataTermSoft prompt tuningTermSynthetic supervisionTermScaling lawsTerm