AI Glossary

State-space model

Also called SSM

What does “State-space model” mean?

State-space model: Handles sequences by updating a compact internal state as each new token arrives, so cost grows roughly in line with input length rather than faster, as it does with attention. Mamba, introduced in 2023, made the approach competitive by letting the state update depend on the input.

State-space model visual guide: Handles sequences by updating a compact internal state as each new token arrives, so cost grows roughly in line with input length rather than faster, as it does with attention. Mamba, introduced in 2023, made the approach competitive by letting the state update depend on the input.
Visual guide Open full size (new tab)

What is an example of “State-space model”?

Some recent models are hybrids that mix Mamba-style layers with attention layers, aiming to handle long inputs more efficiently than a pure transformer.

Keep learning

See also

TransformerTermRecurrent neural networkTermContext windowTermSelf-attentionTerm

Same topic

Long short-term memoryTermSmall language modelTermVariational autoencoderTermVision transformerTermVision-language modelTerm