Modern AI Engineering

Lesson 10.5 · 25 min · video lesson

Rerankers: Re-Scoring Retrieved Results by Relevance

Vector search found 50 "relevant" passages in 20 milliseconds, but the one that actually answers the question is sitting at position 17. How do we get it to the top?

In short: A reranker is a second, more precise model that re-scores a short list of candidates from a fast first-stage search. First-stage retrievers (BM25, bi-encoders) score documents independently of the query, which is fast but loose; a cross-encoder reranker reads the query and each document together and judges relevance much more accurately, at a much higher cost per document. Two-stage retrieval gets most of the accuracy of the slow model at close to the speed of the fast one.

What you will learn

  • What is a Reranker
  • Where a Reranker sits in a search / RAG pipeline
  • The two-stage retrieval idea
  • Why first-stage retrieval is fast but not precise
  • Bi-encoder vs Cross-encoder
  • How a Reranker scores documents step by step
  • The accuracy vs latency and cost trade-off
  • Late-interaction models like ColBERT
  • Real examples of Rerankers
  • Why Rerankers matter for RAG

In this lesson

  1. What is a reranker?
  2. Where a reranker sits: the two-stage retrieval idea
  3. Why first-stage retrieval is fast but not precise
  4. Bi-encoder vs cross-encoder
  5. How a reranker scores documents, step by step
  6. The accuracy vs latency and cost trade-off
  7. Late-interaction models like ColBERT
  8. Real rerankers, and why they matter for RAG
  9. Worked example, step by step
  10. Practice: try it yourself

Key terms in this lesson

Reranker
A model that reorders a short list of retrieved candidates by relevance to the query.
Two-stage retrieval
A cheap wide search for candidates followed by an expensive precise re-scoring.
Bi-encoder
A model that encodes query and document separately into vectors compared by similarity.
Cross-encoder
A model that reads query and document together and outputs a single relevance score.
Precision@k
The fraction of the top k results that are relevant.
Late interaction
Encoding texts separately into token vectors and matching them at scoring time, as in ColBERT.

This lesson is part of the Generative AI Engineering and Complete AI Engineer tracks. See plans and pricing to open the full lesson, its video, labs and quiz, or try the free lesson first (free account needed).

← 10.4 Hybrid Search: Combining Sparse and Dense Retrieval · 10.6 ColBERT: Token-Level Late Interaction for Retrieval →