AI Glossary

Model serving

Also called serving

What does “Model serving” mean?

Model serving: Everything needed to make a trained model available for real use: hosting it, routing requests, scaling up and down with demand, monitoring errors and controlling costs. It is the operational side of AI, as opposed to research and training.

Model serving visual guide: Everything needed to make a trained model available for real use: hosting it, routing requests, scaling up and down with demand, monitoring errors and controlling costs. It is the operational side of AI, as opposed to research and training.
Visual guide Open full size (new tab)

What is an example of “Model serving”?

A model that works well in a lab can still fail as a product if serving is too slow or too expensive.

Keep learning

See also

Inference serverTermBatchingTermCloud AITermInferenceTerm

Same topic

NPUTermModel compressionTermOn-device AITermInterconnectTermONNXTerm