AI Glossary

Mixed-precision training

What does “Mixed-precision training” mean?

Mixed-precision training: Using lower-precision operations where possible and higher precision where numerical stability requires it.

Mixed-precision training visual guide: Using lower-precision operations where possible and higher precision where numerical stability requires it.
Visual guide Open full size (new tab)

What is an example of “Mixed-precision training”?

A training job computes many operations in bfloat16 while keeping selected state in higher precision.

Keep learning

See also

GPUTermComputeTermQuantizationTermData parallelismTermExpert parallelismTerm

Same topic

Bfloat16TermFP8TermModel compressionTermMemory-bound workloadTermModel endpointTerm