AI Glossary

Scaling laws

Also called neural scaling laws

What does “Scaling laws” mean?

Scaling laws: Patterns observed in experiments: a model's loss tends to improve predictably as parameters, training data and compute increase. Researchers use them to estimate how a larger model will perform and how to split a fixed budget between model size and data.

Scaling laws visual guide: Patterns observed in experiments: a model's loss tends to improve predictably as parameters, training data and compute increase. Researchers use them to estimate how a larger model will perform and how to split a fixed budget between model size and data.
Visual guide Open full size (new tab)

What is an example of “Scaling laws”?

DeepMind's Chinchilla research suggested many earlier large models were undertrained, and that model size and training data should grow together.

Keep learning

See also

ComputeTermPre-trainingTermParametersTerm

More from Franklin

On Franklin

AI research explainersPage

Same topic

FLOPsTermSynthetic dataTermReward modelTermTransfer learningTermReinforcement learning from verifiable rewardsTerm