Hybrid search and reranking
Search by exact words and by meaning at once, merge the two lists, then let a slower model reread the best few.
Read first:BM25 keyword searchEmbeddings
Step 1 of 6· Illustrative articles, rankings and reranker scores; the RRF arithmetic is real
Search systems find passages in two main ways. Keyword search, such as BM25, rewards passages that share the question's exact words, so it is good at names, codes and rare terms. Vector search compares meanings, so it can find a page about “paying on a trip” when you asked about “abroad”. Each misses things the other catches. Hybrid search runs both and merges their ranked lists, often with reciprocal rank fusion, which rewards passages that rank well in either list and most of all in both. A reranker then reads the question together with each of the top few passages and reorders them: slower, but more accurate.
Why it matters for your product
Many wrong answers from AI search start here: the right passage never reaches the model. Keyword search catches product names, error codes and clause numbers that embeddings blur; vector search catches paraphrases. In Anthropic's tests on context-enriched chunks, adding BM25 to embedding search cut failed top-20 retrievals from 3.7% to 2.9%, and adding a reranker cut them to 1.9%. The price is a second index and a reranking pass that costs time and money per candidate, so teams rerank a shortlist: Anthropic reranked 150 candidates down to 20.
For engineersShow the maths
RRF(d) = Σ over lists r of 1 ÷ (k + rankᵣ(d)), k = 60
Every list a passage appears in adds a share that shrinks slowly with its rank; a list that does not contain it adds nothing. Only positions are used, never the raw scores, so BM25 scores and cosine similarities never need rescaling.
Worked example: Third in keyword search and second in vector search: 1/63 + 1/62 = 0.0159 + 0.0161 = 0.0320. First and fifth: 1/61 + 1/65 = 0.0318. Agreement edges out a single first place.
relevance(q, p) = σ(w · BERT([CLS] q [SEP] p [SEP])[CLS])
A cross-encoder joins the query and the passage into one input. The vector at the [CLS] position goes through a single-layer network, and the sigmoid σ turns it into the probability that the passage is relevant. Each passage is scored on its own and the list is sorted by that probability.
Worked example: Nogueira and Cho reranked BM25's top 1,000 passages this way on the MS MARCO dev set and raised MRR@10 from 16.7 to 36.5.
cross-encoder passes to compare every pair = n × (n − 1) ÷ 2
Because a cross-encoder must see both texts together, nothing can be precomputed: every pair costs a full model pass. A bi-encoder embeds each text once and compares vectors cheaply.
Worked example: For 10,000 sentences that is about 50 million passes: roughly 65 hours with BERT as a cross-encoder, against about 5 seconds with Sentence-BERT's separate embeddings.
RRF sees only positions, so it cannot tell a close second from a distant one, and nothing can rescue a passage that neither retriever found. Hybrid is not automatically better either: in the DPR paper, adding BM25 to dense retrieval helped on some datasets and hurt on others, so test it on your own queries.
The words you will hear
BM25
Primary sources
- 01Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning MethodsCormack et al. · 2009
- 02The Probabilistic Relevance Framework: BM25 and BeyondRobertson, Zaragoza · 2009
- 03Dense Passage Retrieval for Open-Domain Question AnsweringKarpukhin et al. · 2020
- 04Passage Re-ranking with BERTNogueira, Cho · 2019
- 05Sentence-BERT: Sentence Embeddings using Siamese BERT-NetworksReimers, Gurevych · 2019
- 06Introducing Contextual RetrievalAnthropic · 2024
Connected ideas
Retrieval and search
Retrieval-augmented generation (RAG)
How a model answers from your documents, and shows you exactly where each claim came from.
ExploreRetrieval and search
BM25 keyword search
The decades-old scoring formula inside many search engines, and why repeating a word stops helping.
ExploreRetrieval and search
HNSW vector search
How a stack of sparse-to-dense graphs finds the nearest vectors without checking them all.
ExploreInside the transformer
Encoders vs decoders
The same transformer blocks, trained either to understand a whole text or to write the next word.
ExploreWant this working on your data?
We design and build the systems these ideas power: retrieval, agents, voice and the models behind them. Start with a free discovery call.