Why does a reranker fix RAG’s top result?
Reranking: a cross-encoder reorders the search’s top chunks · Builder · 5 min · rag, embeddings · after Chunking: chunk size, overlap and boundaries in RAG
A reranker is a second model that reads the question and each retrieved chunk together and scores how well the chunk answers it. It reorders the search’s top few, but cannot find a chunk the search left out.
1 Guess first
2 Look
Search, then rerank
1 / 3ask:“What is the daily limit if I don’t pay?” (answer: 1,000)
1 · search: top 3 by cosine (question and chunk embedded apart)
- A✗ Every account starts on the Free plan. The Pro plan costs $20 a month, billed in advance. It allows 50,0000.40
- C✓ address and your API key appears at once. It allows 1,000 requests per day. Requests over the daily limit return0.39
- B✗ requests per day, and support answers within four hours. The Free plan needs no card: sign up with an email0.37
2 · reranker: reads the question and each chunk together
- C✓ address and your API key appears at once. It allows 1,000 requests per day. Requests over the daily limit return-6.4
- B✗ requests per day, and support answers within four hours. The Free plan needs no card: sign up with an email-10.2
- A✗ Every account starts on the Free plan. The Pro plan costs $20 a month, billed in advance. It allows 50,000-10.8
Search alone returns A ✗ · after the reranker: C ✓, which holds the answer
search: all-MiniLM-L6-v2, cosine · reranker: ms-marco-MiniLM-L6-v2, relevance score · measured 2026-10-23
The search ranks the Pro chunk first. The reranker reads each of the top 3 with the question and lifts C, the one with 1,000.
The search shortlists chunks by how they sound; the reranker reads each with the question and reorders the shortlist, but only the shortlist.
The search compares two embeddings made apart: the chunk’s, made once at indexing, and the question’s. Fast enough for millions of chunks. A reranker (a model that scores question–chunk pairs) reads both texts in one pass, so each question word meets each chunk word. Sharper, but it runs once per pair, so it sees only the top few.
4 You are the model
You are the reranker. For four questions, read the search’s top 3 chunks and pick the one ms-marco-MiniLM-L6-v2 scores highest.
you are the reranker · read the question and the search’s top 3, pick the chunk ms-marco-MiniLM-L6-v2 puts first
“What is the daily limit if I don’t pay?”
search: all-MiniLM-L6-v2, cosine · reranker: ms-marco-MiniLM-L6-v2, relevance score · measured 2026-10-23
You are the reranker. Which chunk answers the question?5 Check
Habits that help. Retrieve wide, rerank, keep few: fetch the top 20 to 100 chunks, rerank, paste the best 3 to 5. A reranker cannot restore context a cut removed, so fix the chunking first. It can be wrong too: here it pushed the chunk with Pro’s 50,000 below two without it. Test with real questions.
7 Say it back
8 Mission · 5 min
Copy the free key on Jina’s site (no card) and POST to https://api.jina.ai/v1/rerank with model "jina-reranker-v3.5", the query “How many requests per day does the free plan allow?” and chunks B and C as documents (B: “requests per day, and support answers within four hours. The Free plan needs no card: sign up with an email”; C: “address and your API key appears at once. It allows 1,000 requests per day. Requests over the daily limit return”). Does a bigger reranker still pick B?
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Go deeper
Sources
- Reranker scores: raw relevance logits of cross-encoder/ms-marco-MiniLM-L6-v2 (https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2: BERT, 6 layers, 384 wide, trained on MS MARCO; default activation Identity) on 2026-10-23 by scripts/explain-data/reranking.ts, a plain TS forward pass (pooler + classifier on the shared BERT encoder of scripts/explain-data/semantic-search.ts) that reproduces the model card example 8.607138, -4.320078
- Search scores: cosine similarities with sentence-transformers/all-MiniLM-L6-v2, same script and date; the free-plan scores equal the chunking lesson’s (0.753, 0.637, 0.572)
- The page and its five 20-word chunks are the chunking lesson’s (app/components/explain/lessons/chunking-data.ts); the questions are in app/components/explain/lessons/reranking-data.ts
- Scores used (search cosine → reranker logit, chunks A–E): “don’t pay” A 0.404 → −10.76, B 0.373 → −10.22, C 0.387 → −6.43; carry over C 0.497 → −4.22, B 0.452 → −5.07, D 0.397 → −0.59; error C 0.321 → −9.46, D 0.293 → −7.09; free plan B 0.753 → 7.03, C 0.637 → 4.55, A 0.572 → −1.93; Pro plan A 0.613 → −0.56, C 0.596 → 4.14, B 0.594 → 3.77; cut by paragraph (the chunking lesson’s fix), free plan: the Free paragraph is first for both, cosine 0.780 and reranker 9.46 (next: Pro paragraph 4.33)
- https://sbert.net/examples/cross_encoder/applications/README.html (checked 2026-10-23): “A Cross-Encoder does not produce a sentence embedding”; retrieve e.g. the top 100 with a bi-encoder, then re-rank them with a cross-encoder
- https://jina.ai/reranker/ (checked 2026-10-23): endpoint https://api.jina.ai/v1/rerank (model, query, documents, top_n); recommended text model jina-reranker-v3.5; “New users get an auto-generated API key with free tokens usable across any of our models”; aifree.dev catalog jina-ai-free-api-tokens (verified 2026-10-12): 10M tokens per new API key, no card
Published Oct 23, 2026 · updated Oct 23, 2026