Reranking Model

Reranking is a technique used in search systems or RAG (Retrieval-Augmented Generation) pipelines to improve the relevance of results.

For example, in a RAG system, a user query must be compared against a large number of documents to retrieve the most relevant information. Depending on the use case, these knowledge bases can contain thousands, millions, or even billions of documents. It is therefore essential to have a fast and efficient retrieval mechanism.

To address this need, RAG systems typically rely on vector search.

Vector search and embeddings

Vector search consists of transforming textual content into numerical vectors called embeddings.

These vectors represent the meaning of the text in a multidimensional vector space. In this space, texts with similar meanings are located close to each other.

In this simplified example, each word is represented by a vector. We observe that some words are positioned in similar directions:

• Clovis and King are close
• Climbing and Mountain are close
• some semantic relationships appear as vector transformations

This representation allows search systems to compare the meaning of texts, rather than just their keywords.

Once documents are converted into vectors, they are stored in a vector database. When a query is made, it is also transformed into a vector. The system then compares this vector with those in the database to identify the closest documents.

Limitations of vector search

Although this approach is very effective for large-scale search, it has an important limitation.

Embeddings are a compressed representation of the content of a text. This means that some of the original information is lost when projecting into vector space.

As a result:
documents retrieved at the top of the results are not always the most relevant to precisely answer the user’s question.

Some important documents may appear lower in the ranking, simply because their vector representation is less similar to the query vector.

Limitations related to LLMs and context window

A naive solution would be to increase the number of documents retrieved by vector search.

However, this creates another problem.

LLMs (Large Language Models) have a limited context window, meaning they can only process a limited amount of text at once.

If too many documents are provided to the model:

• response quality may decrease
• important information may be lost
• generation costs may increase

The goal is therefore:

retrieve enough documents to avoid missing relevant information, while keeping only the most relevant ones in the context sent to the LLM.

This is precisely the role of reranking.

Why use a reranker

Reranking consists of re-evaluating the documents retrieved by vector search in order to reorder their ranking.

Instead of relying solely on vector similarity, a specialized model analyzes the query and each document together to estimate their true relevance.

This makes it possible to:

• identify truly useful documents
• filter out less relevant results
• optimize the information sent to the LLM

In many cases, this step leads to:

+20% to +40% improvement in relevance in RAG systems.

The pipeline can be summarized in three main steps:

1️ Search
Vector search retrieves a set of potentially relevant documents.

2️ Re-rank
The reranking model evaluates each document considering the query to produce a more accurate ranking.

3️ Answer
Only the top-ranked documents are sent to the language model to generate the response.

Overall workflow in a RAG pipeline

In a typical RAG pipeline, reranking occurs after the retrieval phase.

The process generally follows these steps:

1. Transform the query into an embedding
2. Perform vector search in the document database
3. Retrieve the top K closest documents
4. Re-evaluate these documents using a reranking model
5. Select the most relevant documents
6. Send these documents as context to the LLM

Model used

In Clovis, we use the model: bge-reranker-v2-m3

It is a cross-encoder, meaning that the query and the document are analyzed together by the model to produce a relevance score.

Feature	Description
Type	Cross Encoder
Input	Query + Document
Output	Relevance score
Languages	Multilingual
Usage	Search / RAG

Unlike an embedding model, the reranker does not produce vectors.
It directly outputs a relevance score between the query and each document.

Example

Query

How does thermal insulation in a house work?

Reranking result

Document	Reranker score
External thermal insulation	0.97
Attic insulation	0.91
Acoustic insulation	0.12

The reranker allows relevant documents to be ranked first.

Use cases

Reranking is particularly useful for:

• RAG pipelines
• document search engines
• internal AI assistants
• question-answering systems over document bases