AI Glossary

Masked language modeling

Also called MLM

What does “Masked language modeling” mean?

Masked language modeling: Training a model to predict hidden tokens from surrounding context.

Masked language modeling visual guide: Training a model to predict hidden tokens from surrounding context.
Visual guide Open full size (new tab)

What is an example of “Masked language modeling”?

A model sees a sentence with a missing word and learns to fill the gap.

Keep learning

See also

Fine-tuningTermInstruction tuningTermKnowledge distillationTermSoft prompt tuningTermContrastive learningTerm

Same topic

Next-token predictionTermSynthetic supervisionTermMomentum in optimizationTermLoss functionTermLoRATerm