AI Glossary

Self-supervised learning

What does “Self-supervised learning” mean?

Self-supervised learning: Learning that creates its own training signal from unlabeled data, typically by hiding part of an input and asking the model to fill it in. Because no human labeling is needed, it can scale to enormous datasets. Large language models learn most of their knowledge this way.

Self-supervised learning visual guide: Learning that creates its own training signal from unlabeled data, typically by hiding part of an input and asking the model to fill it in. Because no human labeling is needed, it can scale to enormous datasets. Large language models learn most of their knowledge this way.
Visual guide Open full size (new tab)

What is an example of “Self-supervised learning”?

Predicting the next word in billions of sentences teaches a language model grammar, facts and style without anyone labeling the text.

Keep learning

See also

Unsupervised learningTermSupervised learningTermPre-trainingTermLarge language modelTerm

Same topic

BERTTermReinforcement learningTermTrainingTermPromptTermTraining dataTerm