AI Glossary

Time per output token

Also called TPOT

What does “Time per output token” mean?

Time per output token: The average elapsed generation time per output token after the first token, under a specified measurement.

Time per output token visual guide: The average elapsed generation time per output token after the first token, under a specified measurement.
Visual guide Open full size (new tab)

What is an example of “Time per output token”?

A benchmark compares the pace at which two models produce the rest of a response.

Keep learning

See also

Inference throughputTermLatencyTermPrefillTermInter-token latencyTermEnd-to-end latencyTerm

Same topic

Continuous batchingTermTime to first tokenTermTensor parallelismTermTPUTermTensor coreTerm