AI Glossary
Top-p
Also called nucleus sampling
What does “Top-p” mean?
Top-p: A sampling setting where the model only chooses from the smallest group of likely next tokens whose probabilities add up to a threshold p, ignoring the long tail of unlikely ones. A related setting, top-k, simply keeps a fixed number of the most likely tokens. Both are usually adjusted alongside, or instead of, temperature.
What is an example of “Top-p”?
With top-p at 0.9, the model picks from candidates covering 90% of the probability and skips the rare oddball words.