R7.4 · LESSON 4 OF 6

Query transformation

Fuse rankings, rerank evidence and evaluate retrieval separately from grounded answer generation.

THE IDEA

Query transformation

Sometimes the user's words are not the best search query. You can rewrite the question, expand it with synonyms, or decompose a compound question into sub-queries. Each costs extra searches (original plus 3 rewrites is 4) and each can change the meaning: a rewrite that drops an identifier or guesses which entity is meant answers a different question. Protect names and identifiers, and evaluate transformed against original queries.

sub-queries: "Pro price", "Team price" 2 searches × 5 results = 10 retrieved; duplicates removed → 8 unique documents (2 repeats) for 1 original + 3 rewrites you run 4 searches

original: "Why did Acme fail E1042" rewrite: "why did acme fail with e1042" accepted: Acme and E1042 are kept evaluate both the transformed and the original query on questions with known answers

Compare the three transformations, then a safe and an unsafe rewrite.

Predict first. Should a rewrite silently replace an ambiguous name with a guessed identity?

Chapter context

RAG combines a retrieval step with a generator that uses retrieved evidence. Better generation cannot reliably repair missing or wrong evidence. Measure retrieval and answer quality as separate stages and connect citations to actual supporting passages.

TRY IT YOURSELF

Part 1. Do it hands-on

Count retrievals and repeats by hand, then write expansion, decomposition, merging and a safe-rewrite guard.

sub-queries: "Pro price", "Team price" 2 searches × 5 results = 10 retrieved; duplicates removed → 8 unique documents (2 repeats) for 1 original + 3 rewrites you run 4 searches

original: "Why did Acme fail E1042" rewrite: "why did acme fail with e1042" accepted: Acme and E1042 are kept evaluate both the transformed and the original query on questions with known answers

retrieved by 2 sub-queries × 5 searches: original + 3 rewrites unique after 3 repeats of 10 silently guess an ambiguous name? (yes/no)

To do: A question is decomposed into 2 sub-queries and each retrieves 5 documents. Documents retrieved before removing duplicates?
To do: You search with the original query plus 3 rewrites. Number of searches (and so the multiple of the single-query cost)?
To do: If 3 of those 10 retrieved documents are duplicates of others, unique documents?
To do: Should a rewrite silently replace an ambiguous name with a guessed identity? Type yes or no

Part 2. Do it in code

expand_query(terms, synonyms): the original terms followed by any synonyms of them (from the dict term → list), without duplicates, order preserved.
decompose(question): split on " and " (case-insensitive) into stripped, non-empty sub-queries.
multi_search(subqueries, results, k): results maps a query to its ranked id list. Return the unique ids from the first k results of each sub-query in order of first appearance.
duplicates(subqueries, results, k): how many retrieved ids (over all sub-queries, first k each) were repeats.
safe_rewrite(original, rewrite, protected): return the rewrite only if every protected token (case-insensitive) that appears in the original also appears in the rewrite; otherwise return the original.
search_cost(n_variants): the number of searches when you run the original query plus n_variants rewrites.

def expand_query(terms, synonyms):
    pass

def decompose(question):
    pass

def multi_search(subqueries, results, k):
    pass

def duplicates(subqueries, results, k):
    pass

def safe_rewrite(original, rewrite, protected):
    pass

def search_cost(n_variants):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Should a rewrite silently replace an ambiguous name with a guessed identity?

ANoBYes

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

RRF(d)=\sum_r\frac1{k+rank_r(d)},\quad MRR=\frac1Q\sum_q\frac1{rank_q(first\ relevant)}.

An exponential-gain convention is $DCG@K=\sum_{i=1}^K(2^{rel_i}-1)/\log_2(i+1)$ and $nDCG=DCG/IDCG$ . Some tools use raw relevance gains instead: do not mix the conventions silently.

A common textbook-scaled single-term BM25 contribution is

IDF(t)\frac{tf(t,d)(k_1+1)}{tf(t,d)+k_1(1-b+b|d|/avgdl)}.

A commonly used nonnegative IDF is $\log[1+(N-df(t)+0.5)/(df(t)+0.5)]$ . Here tf is term count, df document frequency, N corpus document count and avgdl average document length. Engine-specific constant factors, field statistics and length encodings can differ, so these numbers are a teaching model rather than a promise of byte-for-byte engine scores.

Worked example

With k=60, document A has ranks 1 and 3, while B has ranks 2 and 1. A scores 1/61+1/63≈0.0322665 and B scores 1/62+1/61≈0.0325225, so B wins. For exponential gains and relevance order [1,0,2], DCG is 1+3/2=2.5, compared with ideal 3+1/log2(3).

For the stated BM25 variant, a term occurring twice in an average-length document with k1=1.2 has frequency factor 2×2.2/(2+1.2)=1.375 before multiplying by IDF. Increasing count saturates rather than increasing score linearly.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import log2,isclose
A=1/61+1/63
B=1/62+1/61
assert B > A
assert isclose(A,.032266458495966696)
assert isclose(B,.03252247488101534)
dcg=lambda labels: sum((2**rel-1)/log2(i+2) for i,rel in enumerate(labels))
assert dcg([1,0,2]) == 2.5
ndcg=dcg([1,0,2])/dcg([2,1,0])
assert 0 < ndcg < 1
tf,k1,b,length,average=2,1.2,.75,100,100
factor=tf*(k1+1)/(tf+k1*(1-b+b*length/average))
assert isclose(factor,1.375)

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

A retrieved answer can still hallucinate. RRF's k is a fusion constant, not the result-count k in recall@k. Evaluation must state whether relevance gains are linear or exponential.