AI Glossary

Warm replica

What does “Warm replica” mean?

Warm replica: A running service instance that has already loaded the model and can take requests.

Warm replica visual guide: A running service instance that has already loaded the model and can take requests.
Visual guide Open full size (new tab)

What is an example of “Warm replica”?

A deployment keeps one loaded replica ready for traffic.

Keep learning

See also

Model servingTermInference serverTermModel compressionTermAutoscalingTermCold startTerm

Same topic

Model registryTermVector indexTermTPUTermTime to first tokenTermTime per output tokenTerm