AI Glossary

End-of-sequence token

Also called EOS token

What does “End-of-sequence token” mean?

End-of-sequence token: A token that signals the intended end of a generated sequence.

End-of-sequence token visual guide: A token that signals the intended end of a generated sequence.
Visual guide Open full size (new tab)

What is an example of “End-of-sequence token”?

A decoder stops after the model produces its EOS token.

Keep learning

See also

TokenizerTermTokenTermEmbeddingsTermSentencePieceTermSpecial tokenTerm

Same topic

Token probabilityTermVocabulary sizeTermFeed-forward networkTermEncoder-decoder modelTermFlashAttentionTerm