AI Glossary

Inference

What does “Inference” mean?

Inference: The stage where an already-trained model is put to work: it receives a new input and produces an output. No learning happens during ordinary inference; the model simply applies what it learned in training. Every chatbot reply you receive is a round of inference.

Inference visual guide: The stage where an already-trained model is put to work: it receives a new input and produces an output. No learning happens during ordinary inference; the model simply applies what it learned in training. Every chatbot reply you receive is a round of inference.
Visual guide Open full size (new tab)

What is an example of “Inference”?

Training a model happens once, but inference runs millions of times as people use it, which is why running costs matter.

Keep learning

See also

TrainingTermPredictionTermInference costTermModelTerm

Same topic

AI energy useTermInference serverTermLatencyTermModel servingTermLarge language modelTerm