AI Glossary

High-bandwidth memory

Also called HBM

What does “High-bandwidth memory” mean?

High-bandwidth memory: HBM stacks memory chips vertically and places them right beside the processor, so data moves between them much faster than with ordinary memory. Large AI models constantly shuffle huge amounts of data, so memory speed often limits performance as much as raw compute does.

High-bandwidth memory visual guide: HBM stacks memory chips vertically and places them right beside the processor, so data moves between them much faster than with ordinary memory. Large AI models constantly shuffle huge amounts of data, so memory speed often limits performance as much as raw compute does.
Visual guide Open full size (new tab)

What is an example of “High-bandwidth memory”?

HBM supply, from makers such as SK hynix, Samsung and Micron, has been a bottleneck in producing high-end AI chips.

Keep learning

See also

GPUTermGPU memoryTermAI acceleratorTermInference serverTerm

Same topic

InterconnectTermHugging FaceTermHyperscalerTermFLOPsTermEdge AITerm