AI Glossary

Autoscaling

What does “Autoscaling” mean?

Autoscaling: Adjusting running compute capacity in response to demand or operating signals.

Autoscaling visual guide: Adjusting running compute capacity in response to demand or operating signals.
Visual guide Open full size (new tab)

What is an example of “Autoscaling”?

A model service adds replicas when the request backlog grows.

Keep learning

See also

Model servingTermInference serverTermModel compressionTermModel fallbackTermInference queueTerm

Same topic

Cold startTermWarm replicaTermBatchingTermApproximate nearest neighborsTermBfloat16Term