R1.4 · LESSON 4 OF 6

L2 distance

Rank documents by dot product, cosine and L2 distance and explain the role of normalization and model choice.

THE IDEA

Euclidean distance

L2 distance measures how far apart two points are: √Σ(aᵢ − bᵢ)². Unlike similarities, smaller is closer. Many indexes work with the squared distance, which avoids a square root and gives the same ranking. For unit vectors the squared distance is 2 − 2·cosine, so cosine, dot product and L2 agree on the order once vectors are normalised.

length of document B10angle of document B (degrees)45length of document A (same direction as q)1

document	vector	L2 distance to q	rank
A	[1, 0]	0	1
B	[7.07, 7.07]	9.32	2

L2 distance: how far apart the points are SMALLER is better (0 = the same point)

q = [1, 0], B = [10, 10]: squared L2 = 81 + 100 = 181, L2 = 13.45 rankings by L2 and by squared L2 are identical (the square root keeps the order of non-negative numbers)

Distances are smaller-is-closer, the opposite direction to similarities. Some indexes report squared distances to save the square root.

Distances to q = [1, 0].

Predict first. Does taking a square root change the order of non-negative squared distances?

Chapter context

Semantic search represents queries and documents as vectors in a compatible space. Good retrieval depends on the encoder, its training objective and the candidate data; choosing a distance formula alone does not create semantic relevance.

TRY IT YOURSELF

Part 1. Do it hands-on

Compute distances by hand, then write distance ranking and verify the unit-vector identity.

length of document B10angle of document B (degrees)45length of document A (same direction as q)1

document	vector	L2 distance to q	rank
A	[1, 0]	0	1
B	[7.07, 7.07]	9.32	2

L2 distance: how far apart the points are SMALLER is better (0 = the same point)

q = [1, 0], B = [10, 10]: squared L2 = 81 + 100 = 181, L2 = 13.45 rankings by L2 and by squared L2 are identical (the square root keeps the order of non-negative numbers)

Distances are smaller-is-closer, the opposite direction to similarities. Some indexes report squared distances to save the square root.

squared L2 (q, B) L2 (2 decimals) unit vectors, cosine 0.7071: squared L2

To do: Squared L2 distance between q = [1, 0] and B = [10, 10]: (1−10)² + (0−10)²
To do: The L2 distance itself (2 decimals)?
To do: For two UNIT vectors, squared L2 = 2 − 2·cosine. If the cosine is 0.7071 the squared distance is (4 decimals)?

Part 2. Do it in code

l2(a, b) and squared_l2(a, b): Euclidean distance and its square.
rank_by_l2(q, docs, squared=False): indices from nearest to farthest (smaller distance first, ties: lower index first).
same_order_squared(q, docs): True if ranking by squared L2 equals ranking by L2.
expand_squared(a, b): |a|² + |b|² − 2 a·b, which equals the squared distance.
unit_vector_relation(a, b): for UNIT vectors return (squared_l2(a, b), 2 − 2·cosine(a, b)); the two numbers must agree.

import numpy as np

def l2(a, b):
    pass

def squared_l2(a, b):
    pass

def rank_by_l2(q, docs, squared=False):
    pass

def same_order_squared(q, docs):
    pass

def expand_squared(a, b):
    pass

def unit_vector_relation(a, b):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Does taking a square root change the order of nonnegative squared distances?

ANoBYes

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

\cos(q,x)=\frac{q^Tx}{\|q\|\|x\|},\quad \|q-x\|^2=\|q\|^2+\|x\|^2-2q^Tx.

For unit-normalized vectors, squared L2 is $2-2q^Tx$ , so their rankings agree under the appropriate direction. Zero vectors require a defined handling policy.

Worked example

Let q=[1,0], A=[1,0] and B=[10,10]. Raw dot products are 1 and 10, so B ranks first by dot product. Cosines are 1 and 1/sqrt(2), so A ranks first by cosine. This is a deliberate example of magnitude changing the retrieval objective.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import sqrt,isclose
q,a,b=[1,0],[1,0],[10,10]
dot=lambda u,v: sum(x*y for x,y in zip(u,v))
norm=lambda u: sqrt(dot(u,u))
assert dot(q,b) > dot(q,a)
assert dot(q,a)/(norm(q)*norm(a)) > dot(q,b)/(norm(q)*norm(b))
assert isclose(dot(q,b)/(norm(q)*norm(b)),1/sqrt(2))
assert 128*4 == 512

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Do not mix embedding spaces or metric directions. Normalize only when it matches the model and search objective. Similarity is not factual verification.