AI Glossary

End-to-end latency

Also called E2E latency

What does “End-to-end latency” mean?

End-to-end latency: The time from sending a request to receiving the complete response.

End-to-end latency visual guide: The time from sending a request to receiving the complete response.
Visual guide Open full size (new tab)

What is an example of “End-to-end latency”?

A summarization call includes queueing, model processing, and return time.

Keep learning

See also

Inference throughputTermLatencyTermPrefillTermInter-token latencyTermTime per output tokenTerm

Same topic

Continuous batchingTermPaged attentionTermEuclidean distanceTermEdge AITermExpert parallelismTerm