AI Glossary

Top-k sampling

What does “Top-k sampling” mean?

Top-k sampling: Sampling from only the k tokens with the highest predicted probabilities at a generation step.

Top-k sampling visual guide: Sampling from only the k tokens with the highest predicted probabilities at a generation step.
Visual guide Open full size (new tab)

What is an example of “Top-k sampling”?

A model draws from its twenty most likely next tokens.

Keep learning

See also

DecodingTermTemperatureTermTop-pTermGreedy decodingTermBeam searchTerm

Same topic

TransformerTermToken probabilityTermVariational autoencoderTermTest-time computeTermVerifier modelTerm