AI Glossary

Batch API

Also called batch processing

What does “Batch API” mean?

Batch API: An option many providers offer for submitting a large set of requests to be processed in the background instead of in real time. Results come back later, often within hours, and batch pricing is commonly lower than standard requests. It suits work where nobody is waiting on an immediate answer.

Batch API visual guide: An option many providers offer for submitting a large set of requests to be processed in the background instead of in real time. Results come back later, often within hours, and batch pricing is commonly lower than standard requests. It suits work where nobody is waiting on an immediate answer.
Visual guide Open full size (new tab)

What is an example of “Batch API”?

Tagging a backlog of 50,000 customer reviews overnight is a good fit for a batch API rather than live calls.

Keep learning

See also

APITermInference costTermRate limitTermBatchingTerm

Same topic

Chain of thoughtTermChunkingTermContext windowTermEmbeddingsTermFew-shot promptingTerm