AI Glossary

Checkpoint

What does “Checkpoint” mean?

Checkpoint: A saved snapshot of a model's weights at a particular point in training, often together with the optimizer's state. Checkpoints let teams resume after crashes, compare progress over time and roll back if later training makes the model worse.

Checkpoint visual guide: A saved snapshot of a model's weights at a particular point in training, often together with the optimizer's state. Checkpoints let teams resume after crashes, compare progress over time and roll back if later training makes the model worse.
Visual guide Open full size (new tab)

What is an example of “Checkpoint”?

EleutherAI's Pythia project published many intermediate checkpoints so researchers could study how abilities develop during training.

Keep learning

See also

EpochTermPre-trainingTermFine-tuningTerm

Same topic

ComputeTermBatch sizeTermCurriculum learningTermBackpropagationTermData labelingTerm