R7.2 · LESSON 2 OF 6

Hybrid fusion

Fuse rankings, rerank evidence and evaluate retrieval separately from grounded answer generation.

THE IDEA

Hybrid fusion

To combine lexical and dense results you cannot just add their scores: BM25 might range up to 20 while a cosine is between −1 and 1, so the larger scale decides. Reciprocal rank fusion sidesteps the issue by using only positions: score = Σ 1/(k + rank). With k = 60, a document ranked 1st and 3rd scores 0.0323, one ranked 2nd and 1st scores 0.0325, and wins. Alternatively normalise scores and weight them, with a policy you can justify.

k60A rank in lexical list1A rank in dense list3B rank in lexical list2B rank in dense list1

document	ranks	RRF score
A	1, 3	1/61 + 1/63 = 0.032266
B	2, 1	1/62 + 1/61 = 0.032522

fused winner: B (rank fusion uses positions, so BM25 scores like 12.0 and cosines like 0.8 never need to be compared) adding raw scores 0.80 + 12.0: BM25 supplies 0.938 of the sum

Change ranks and k.

Predict first. Can raw cosine and BM25 scores be added without considering scale?

Chapter context

RAG combines a retrieval step with a generator that uses retrieved evidence. Better generation cannot reliably repair missing or wrong evidence. Measure retrieval and answer quality as separate stages and connect citations to actual supporting passages.

TRY IT YOURSELF

Part 1. Do it hands-on

Fuse two rankings by hand, then implement rank fusion and normalised weighting.

k60A rank in lexical list1A rank in dense list3B rank in lexical list2B rank in dense list1

document	ranks	RRF score
A	1, 3	1/61 + 1/63 = 0.032266
B	2, 1	1/62 + 1/61 = 0.032522

fused winner: B (rank fusion uses positions, so BM25 scores like 12.0 and cosines like 0.8 never need to be compared) adding raw scores 0.80 + 12.0: BM25 supplies 0.938 of the sum

RRF of A (ranks 1 and 3) RRF of B (ranks 2 and 1) fused winner (A or B) BM25 share of 0.80 + 12.0

To do: Reciprocal rank fusion with k = 60: score = Σ 1/(k + rank). Document A is ranked 1st by one system and 3rd by the other. Score (4 decimals)?
To do: Document B is ranked 2nd and 1st. Score (4 decimals)?
To do: Which document is ranked first after fusion? Type A or B
To do: Adding a raw cosine score 0.80 to a raw BM25 score 12.0: what share of the sum comes from BM25 (3 decimals)? This is why scales matter

Part 2. Do it in code

rrf_scores(rankings, k=60): rankings is a list of ranked id lists (best first). Each id scores the sum of 1/(k + rank) over the lists it appears in (rank starts at 1). Return a dict.
fuse(rankings, k=60): ids sorted by fused score, best first (ties: smaller id).
ranking_from_scores(scores): ids sorted by score, highest first (ties: smaller id).
min_max(scores): rescale a dict of scores to 0..1 ((s − min) / (max − min)); if all scores are equal return 0.0 for each.
weighted_fusion(scores_a, scores_b, w): after min-max normalising both, w·a + (1 − w)·b for every id in either dict (a missing score counts as 0); return the ids best first.

def rrf_scores(rankings, k=60):
    pass

def fuse(rankings, k=60):
    pass

def ranking_from_scores(scores):
    pass

def min_max(scores):
    pass

def weighted_fusion(scores_a, scores_b, w):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Can raw cosine and BM25 scores be added without considering scale?

ANoBYes

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

RRF(d)=\sum_r\frac1{k+rank_r(d)},\quad MRR=\frac1Q\sum_q\frac1{rank_q(first\ relevant)}.

An exponential-gain convention is $DCG@K=\sum_{i=1}^K(2^{rel_i}-1)/\log_2(i+1)$ and $nDCG=DCG/IDCG$ . Some tools use raw relevance gains instead: do not mix the conventions silently.

A common textbook-scaled single-term BM25 contribution is

IDF(t)\frac{tf(t,d)(k_1+1)}{tf(t,d)+k_1(1-b+b|d|/avgdl)}.

A commonly used nonnegative IDF is $\log[1+(N-df(t)+0.5)/(df(t)+0.5)]$ . Here tf is term count, df document frequency, N corpus document count and avgdl average document length. Engine-specific constant factors, field statistics and length encodings can differ, so these numbers are a teaching model rather than a promise of byte-for-byte engine scores.

Worked example

With k=60, document A has ranks 1 and 3, while B has ranks 2 and 1. A scores 1/61+1/63≈0.0322665 and B scores 1/62+1/61≈0.0325225, so B wins. For exponential gains and relevance order [1,0,2], DCG is 1+3/2=2.5, compared with ideal 3+1/log2(3).

For the stated BM25 variant, a term occurring twice in an average-length document with k1=1.2 has frequency factor 2×2.2/(2+1.2)=1.375 before multiplying by IDF. Increasing count saturates rather than increasing score linearly.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import log2,isclose
A=1/61+1/63
B=1/62+1/61
assert B > A
assert isclose(A,.032266458495966696)
assert isclose(B,.03252247488101534)
dcg=lambda labels: sum((2**rel-1)/log2(i+2) for i,rel in enumerate(labels))
assert dcg([1,0,2]) == 2.5
ndcg=dcg([1,0,2])/dcg([2,1,0])
assert 0 < ndcg < 1
tf,k1,b,length,average=2,1.2,.75,100,100
factor=tf*(k1+1)/(tf+k1*(1-b+b*length/average))
assert isclose(factor,1.375)

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

A retrieved answer can still hallucinate. RRF's k is a fusion constant, not the result-count k in recall@k. Evaluation must state whether relevance gains are linear or exponential.