R1.1 · LESSON 1 OF 6

Query and document encoders

Rank documents by dot product, cosine and L2 distance and explain the role of normalization and model choice.

THE IDEA

Turning text into points to search

Embedding retrieval converts a query and a set of documents into vectors, then returns the documents whose vectors are closest to the query's. For this to mean anything, the query and documents must be encoded by compatible models: ones trained together so that their coordinates share a meaning. Some models also treat the two roles differently, for example by requiring a prefix on queries and another on documents.

A toy encoder counts five words. The second "encoder" has the same size (5 numbers) but lists the words in another order, so its coordinates mean something else.

document	vector	dot with the query
paris is the capital of france	[1, 1, 1, 0, 0]	2
rome is the capital of italy	[0, 1, 0, 0, 1]	1
pizza in rome	[0, 0, 0, 1, 1]	0

query vector: [0, 1, 1, 0, 0] best match: "paris is the capital of france" matching dimensions made the arithmetic possible, not the answer meaningful

Retrieval models also give queries and documents different roles; some require prefixes such as query: and passage: . Follow the contract of the model you chose.

Same size, different encoders: see what happens to the ranking.

Predict first. Can embeddings from unrelated encoders be compared just because their dimensions match?

Chapter context

Semantic search represents queries and documents as vectors in a compatible space. Good retrieval depends on the encoder, its training objective and the candidate data; choosing a distance formula alone does not create semantic relevance.

TRY IT YOURSELF

Part 1. Do it hands-on

Apply a model's role contract and see a mismatched encoder break a ranking.

A toy encoder counts five words. The second "encoder" has the same size (5 numbers) but lists the words in another order, so its coordinates mean something else.

document	vector	dot with the query
paris is the capital of france	[1, 1, 1, 0, 0]	2
rome is the capital of italy	[0, 1, 0, 0, 1]	1
pizza in rome	[0, 0, 0, 1, 1]	0

query vector: [0, 1, 1, 0, 0] best match: "paris is the capital of france" matching dimensions made the arithmetic possible, not the answer meaningful

Retrieval models also give queries and documents different roles; some require prefixes such as query: and passage: . Follow the contract of the model you chose.

meaningful across unrelated models? (yes/no) text to encode for the query capital of France multiplications for two 384-d vectors

To do: Two different, unrelated embedding models both output 384-number vectors. Because the sizes match, is the dot product of a vector from one with a vector from the other a meaningful relevance score? Type yes or no
To do: A model's contract says queries must start with "query: " and documents with "passage: ". Type the exact string you would encode for the query: capital of France
To do: Comparing two 384-dimensional vectors with a dot product takes how many multiplications?

Part 2. Do it in code

role_text(text, role, prefixes): prefixes maps a role ("query" or "document") to its required prefix. Return prefixes[role] + text. KeyError-like error (ValueError) for an unknown role.
bow_encode(text, vocab): a vector of word counts in the order of the vocab list (lower-case, split on whitespace); words outside the vocabulary are ignored.
check_compatible(vec_a, vec_b, model_a, model_b): return True only if the dimensions are equal AND model_a == model_b. Raise ValueError when the dimensions differ.
rank(query_vec, doc_vecs): document indices sorted by dot product, best first (ties: lower index first).
rank_with_mismatched_encoders(query, docs, vocab_a, vocab_b): encode the query with vocab_a but the documents with vocab_b (a permutation of the same words) and rank; this shows how comparing vectors from unrelated encoders gives a different, wrong order than encoding both with vocab_a.

import numpy as np

def role_text(text, role, prefixes):
    pass

def bow_encode(text, vocab):
    pass

def check_compatible(vec_a, vec_b, model_a, model_b):
    pass

def rank(query_vec, doc_vecs):
    pass

def rank_with_mismatched_encoders(query, docs, vocab_a, vocab_b):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Can embeddings from unrelated encoders be compared just because their dimensions match?

AYesBNo

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

\cos(q,x)=\frac{q^Tx}{\|q\|\|x\|},\quad \|q-x\|^2=\|q\|^2+\|x\|^2-2q^Tx.

For unit-normalized vectors, squared L2 is $2-2q^Tx$ , so their rankings agree under the appropriate direction. Zero vectors require a defined handling policy.

Worked example

Let q=[1,0], A=[1,0] and B=[10,10]. Raw dot products are 1 and 10, so B ranks first by dot product. Cosines are 1 and 1/sqrt(2), so A ranks first by cosine. This is a deliberate example of magnitude changing the retrieval objective.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import sqrt,isclose
q,a,b=[1,0],[1,0],[10,10]
dot=lambda u,v: sum(x*y for x,y in zip(u,v))
norm=lambda u: sqrt(dot(u,u))
assert dot(q,b) > dot(q,a)
assert dot(q,a)/(norm(q)*norm(a)) > dot(q,b)/(norm(q)*norm(b))
assert isclose(dot(q,b)/(norm(q)*norm(b)),1/sqrt(2))
assert 128*4 == 512

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Do not mix embedding spaces or metric directions. Normalize only when it matches the model and search objective. Similarity is not factual verification.