AI Glossary

Feed-forward network

Also called FFN

What does “Feed-forward network” mean?

Feed-forward network: A network that sends information forward through layers without recurrent connections.

Feed-forward network visual guide: A network that sends information forward through layers without recurrent connections.
Visual guide Open full size (new tab)

What is an example of “Feed-forward network”?

A transformer applies a small feed-forward network to each token after attention.

Keep learning

See also

TransformerTermAttention mechanismTermModelTermSparse attentionTermFlashAttentionTerm

Same topic

Multilayer perceptronTermEnd-of-sequence tokenTermFrontier modelTermEncoder-decoder modelTermGenerative adversarial networkTerm