AI Glossary

Quantization

What does “Quantization” mean?

Quantization: Storing a model's numbers with fewer bits, for example 8-bit or 4-bit instead of 16-bit. It shrinks the model and speeds it up, usually with a small loss in quality, and is one of the most common ways to fit large models onto modest hardware.

Quantization visual guide: Storing a model's numbers with fewer bits, for example 8-bit or 4-bit instead of 16-bit. It shrinks the model and speeds it up, usually with a small loss in quality, and is one of the most common ways to fit large models onto modest hardware.
Visual guide Open full size (new tab)

What is an example of “Quantization”?

Quantized versions of open-weight models let hobbyists run capable chatbots on a single consumer graphics card.

Keep learning

See also

Model compressionTermPruningTermGPU memoryTermOn-device AITerm

Same topic

Edge AITermKnowledge distillationTermLoRATermNPUTermQLoRATerm