AI Glossary

Next-token prediction

What does “Next-token prediction” mean?

Next-token prediction: Training or running a model to predict the next token in a sequence.

Next-token prediction visual guide: Training or running a model to predict the next token in a sequence.
Visual guide Open full size (new tab)

What is an example of “Next-token prediction”?

A model reads the beginning of a sentence and predicts likely continuations.

Keep learning

See also

Fine-tuningTermInstruction tuningTermKnowledge distillationTermContrastive learningTermMasked language modelingTerm

Same topic

Preference datasetTermSynthetic supervisionTermOptimizerTermMomentum in optimizationTermOverfittingTerm