AI GitHub
Sentence Transformers for embeddings and reranking
Encode inputs for semantic search or score query-document pairs with a cross-encoder.
Key Takeaways
- The library supports embedding models that encode inputs separately and cross-encoders that score pairs together.
- Read the checkpoint's instructions for encoding queries and passages.
- Retrieve candidates with an embedding model and rerank them with a cross-encoder.
- The model determines language support, input limits and expected formatting.
How it works
The library supports embedding models that encode inputs separately and cross-encoders that score pairs together. Those modes serve different stages of a search pipeline.
What you need
Read the checkpoint's instructions for encoding queries and passages. Choose a similarity measure that matches the model and apply it consistently when building and querying the index.
What to try first
Retrieve candidates with an embedding model and rerank them with a cross-encoder. Compare quality and time per query at two candidate counts.
Compare short and long inputs against the same relevance judgments. Truncation can affect later terms in a query or document differently across checkpoints.
Where to look for failures
The model determines language support, input limits and expected formatting. Use a matched query and document encoding recipe when the model documents one.
Related Franklin pages
Check whether the task needs a library inside the app or a separate model endpoint. Preserve query formatting, checkpoint and vector dimensions when comparing either deployment.
- Expose supported embedding and reranking models through a dedicated inference service. Text Embeddings Inference for serving embedding models
- Score query-document pairs with supported Mixedbread reranking checkpoints. mxbai-rerank for ranking search candidates
For the role of the encoder in a search stack, read Franklin’s model guide. Embedding models
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗