AI Glossary

Loss function

Also called cost function, objective

What does “Loss function” mean?

Loss function: The formula that measures how wrong a model's predictions are, reduced to a single number. Training is essentially the process of adjusting the model so that this number goes down.

Loss function visual guide: The formula that measures how wrong a model's predictions are, reduced to a single number. Training is essentially the process of adjusting the model so that this number goes down.
Visual guide Open full size (new tab)

What is an example of “Loss function”?

Language models are usually trained with cross-entropy loss, which penalizes the model for giving low probability to the actual next token.

Keep learning

See also

Gradient descentTermBackpropagationTermOverfittingTermLearning rateTerm

Same topic

RegularizationTermUnderfittingTermLoRATermPre-trainingTermQLoRATerm