AI Glossary

Grouped-query attention

Also called GQA

What does “Grouped-query attention” mean?

Grouped-query attention: An attention design where groups of query heads share key and value heads.

Grouped-query attention visual guide: An attention design where groups of query heads share key and value heads.
Visual guide Open full size (new tab)

What is an example of “Grouped-query attention”?

A serving system stores fewer key and value heads than a full multi-head attention design.

Keep learning

See also

TransformerTermAttention mechanismTermModelTermRotary position embeddingTermMulti-query attentionTerm

Same topic

Sliding-window attentionTermLatent spaceTermGreedy decodingTermLog probabilityTermGPTTerm