AI Glossary

Adam optimizer

Also called adaptive moment estimation

What does “Adam optimizer” mean?

Adam optimizer: An optimizer family that adapts update sizes using running estimates of gradients and squared gradients.

Adam optimizer visual guide: An optimizer family that adapts update sizes using running estimates of gradients and squared gradients.
Visual guide Open full size (new tab)

What is an example of “Adam optimizer”?

A training run uses AdamW, a variant with decoupled weight decay.

Keep learning

See also

TrainingTermRegularizationTermLearning rateTermOptimizerTermStochastic gradient descentTerm

Same topic

Momentum in optimizationTermWeight decayTermAdapter layerTermActive learningTermBackpropagationTerm