AI Glossary

Layer normalization

Also called layer norm

What does “Layer normalization” mean?

Layer normalization: Normalizing a layer's activations within each example, usually across its feature dimension.

Layer normalization visual guide: Normalizing a layer's activations within each example, usually across its feature dimension.
Visual guide Open full size (new tab)

What is an example of “Layer normalization”?

A transformer applies layer normalization separately to each token's hidden state.

Keep learning

See also

Deep learningTermTrainingTermOverfittingTermBatch normalizationTermRMS normalizationTerm

Same topic

Weight initializationTermLearning rateTermKnowledge distillationTermLearning rate scheduleTermInstruction tuningTerm