AI Glossary

Time to first token

Also called TTFT

What does “Time to first token” mean?

Time to first token: The time from sending a request until the first output token arrives. Depending on the measurement boundary, it can include queueing, network time, and prompt processing.

Time to first token visual guide: The time from sending a request until the first output token arrives. Depending on the measurement boundary, it can include queueing, network time, and prompt processing.
Visual guide Open full size (new tab)

What is an example of “Time to first token”?

One assistant starts its answer sooner even though another writes the rest of the answer faster.

Keep learning

See also

LatencyTermPrefillTermInference throughputTermStreamingTerm

Same topic

TPUTermTime per output tokenTermVector indexTermTensor parallelismTermWarm replicaTerm