AI Glossary

Transformer

What does “Transformer” mean?

Transformer: Introduced in the 2017 paper "Attention Is All You Need," this neural network architecture processes a whole sequence at once and uses attention to work out how each piece of input relates to every other piece. Most modern language, image and audio models are built on it.

Transformer visual guide: Introduced in the 2017 paper "Attention Is All You Need," this neural network architecture processes a whole sequence at once and uses attention to work out how each piece of input relates to every other piece. Most modern language, image and audio models are built on it.
Visual guide Open full size (new tab)

What is an example of “Transformer”?

GPT, Claude, Gemini and Llama are all transformer-based, which is why the architecture is often called the backbone of today's generative AI.

Keep learning

See also

Attention mechanismTermSelf-attentionTermEncoderTermDecoderTerm

More from Franklin

On Franklin

AI research explainersPage

Same topic

BERTTermDense modelTermEncoder-decoder modelTermGPTTermLong short-term memoryTerm