Embedding model
Turns text into a list of numbers that captures its meaning, so similar ideas can be found by math.
What it is
Embedding models convert a sentence, paragraph or document into a vector. Texts with similar meaning end up close together, which is the basis of semantic search, recommendation, clustering and retrieval-augmented generation (RAG).
How it works
A transformer encoder reads the text and pools its output into one fixed-length vector. Contrastive training on pairs such as question and answer pulls related texts together and pushes unrelated ones apart. Vectors are stored in a vector database and compared with cosine similarity.
What it is good for
- Semantic search over documents
- Retrieval for RAG chatbots
- Deduplication and clustering
- Recommendations
Limits to know
- Meaning is compressed, so exact keywords can be missed
- Changing models means re-embedding everything
- Domain-specific text may need a tuned model
