AI GitHub
rerankers for comparing relevance models
Use a common Python interface to try different reranking implementations.
Key Takeaways
- The library wraps several reranker types behind a common ranking interface.
- Check the installation options for the backend you select.
- Save candidate lists from a known search run, then rerank them with two documented backends.
- Scores from different backends are not necessarily calibrated to the same scale.
How it works
The library wraps several reranker types behind a common ranking interface. It lets an experiment change the reranker while preserving the same retrieval inputs.
What you need
Check the installation options for the backend you select. Some rankers need local weights and additional libraries, while others depend on a remote service.
What to try first
Save candidate lists from a known search run, then rerank them with two documented backends. Compare the change in result order and runtime.
Record the scores and rank changes for a few borderline examples. Differences near a cutoff can matter more than agreement on the obvious results.
Where to look for failures
Scores from different backends are not necessarily calibrated to the same scale. Compare rankings using judged examples before introducing a shared score threshold.
Related Franklin pages
Save the first-stage retrieval results and reuse them for reranking tests. A shared candidate set shows whether changes come from the scoring model or from which documents reached it.
- Encode inputs for semantic search or score query-document pairs with a cross-encoder. Sentence Transformers for embeddings and reranking
- Score query-document pairs with supported Mixedbread reranking checkpoints. mxbai-rerank for ranking search candidates
For the role of the encoder in a search stack, read Franklin’s model guide. Embedding models
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗