AI Glossary

Decoding

Also called token decoding

What does “Decoding” mean?

Decoding: The process of generating output from a model's token predictions. A decoding policy picks the next token from the predicted distribution and repeats until a stop condition.

Decoding visual guide: The process of generating output from a model's token predictions. A decoding policy picks the next token from the predicted distribution and repeats until a stop condition.
Visual guide Open full size (new tab)

What is an example of “Decoding”?

A text generator samples the next word fragment at each step until it reaches its end token.

Keep learning

See also

TokenTermAutoregressive modelTermTemperatureTermSpeculative decodingTerm

Same topic

Beam searchTermBest-of-N samplingTermGreedy decodingTermInference throughputTermKV cacheTerm