AI Glossary

Learning rate

What does “Learning rate” mean?

Learning rate: The setting that controls how big a step the optimizer takes each time it updates the model's weights. If it is too high, training can become unstable or diverge. If it is too low, learning is slow or stalls. Many training runs change it over time using a schedule.

Learning rate visual guide: The setting that controls how big a step the optimizer takes each time it updates the model's weights. If it is too high, training can become unstable or diverge. If it is too low, learning is slow or stalls. Many training runs change it over time using a schedule.
Visual guide Open full size (new tab)

What is an example of “Learning rate”?

A common schedule starts with a short warm-up, then gradually lowers the learning rate so the model settles into a good solution.

Keep learning

See also

Gradient descentTermHyperparameterTermBatch sizeTermLoss functionTerm

Same topic

BackpropagationTermLoRATermKnowledge distillationTermInstruction tuningTermOverfittingTerm