AI Glossary

Max tokens

Also called max output tokens

What does “Max tokens” mean?

Max tokens: A request setting that caps how many tokens the model may generate in its reply. When the cap is hit, the output stops, even mid-sentence. It helps control cost and response length, but it does not make the model plan a shorter answer.

Max tokens visual guide: A request setting that caps how many tokens the model may generate in its reply. When the cap is hit, the output stops, even mid-sentence. It helps control cost and response length, but it does not make the model plan a shorter answer.
Visual guide Open full size (new tab)

What is an example of “Max tokens”?

Setting max tokens too low for a long report can leave you with an answer that cuts off halfway through a section.

Keep learning

See also

TokenTermContext windowTermInference costTerm

Same topic

TemperatureTermPrompt cachingTermLatencyTermPrompt engineeringTermKnowledge cutoffTerm