AI Glossary

Gradient descent

What does “Gradient descent” mean?

Gradient descent: This optimization method repeatedly nudges a model's parameters in whichever direction reduces the loss the most. Practical versions, such as stochastic gradient descent and the Adam optimizer, estimate that direction from small batches of data at a time.

Gradient descent visual guide: This optimization method repeatedly nudges a model's parameters in whichever direction reduces the loss the most. Practical versions, such as stochastic gradient descent and the Adam optimizer, estimate that direction from small batches of data at a time.
Visual guide Open full size (new tab)

What is an example of “Gradient descent”?

Picture walking downhill in thick fog: you feel which way the ground slopes under your feet, take a step that way, and repeat.

Keep learning

See also

BackpropagationTermLearning rateTermLoss functionTermBatch sizeTerm

Same topic

AlgorithmTermHyperparameterTermFine-tuningTermInstruction tuningTermEpochTerm