R1.3 · LESSON 3 OF 6

Cosine similarity

Rank documents by dot product, cosine and L2 distance and explain the role of normalization and model choice.

THE IDEA

Cosine similarity

Cosine similarity divides the dot product by both lengths: cos = q·d / (|q||d|). It is 1 for identical directions, 0 for perpendicular vectors and −1 for opposite ones, and it ignores length. If every vector is first normalised to length 1, the plain dot product equals the cosine, so a dot-product index can do cosine ranking.

length of document B10angle of document B (degrees)45length of document A (same direction as q)1

document	vector	cosine with q	rank
A	[1, 0]	1	1
B	[7.07, 7.07]	0.707	2

cosine: only the angle matters; lengths are divided out larger is better (1 = same direction)

Slide B's length: its cosine does not move. Only the angle matters. If all vectors are normalised to length 1, the dot product equals the cosine, so one fast dot-product index can serve cosine ranking.

Same vectors, now ranked by cosine: the order flips.

Predict first. What is the cosine of two identical nonzero vectors?

Chapter context

Semantic search represents queries and documents as vectors in a compatible space. Good retrieval depends on the encoder, its training objective and the candidate data; choosing a distance formula alone does not create semantic relevance.

TRY IT YOURSELF

Part 1. Do it hands-on

Compute cosines by hand, then write normalisation, cosine and cosine ranking.

length of document B10angle of document B (degrees)45length of document A (same direction as q)1

document	vector	cosine with q	rank
A	[1, 0]	1	1
B	[7.07, 7.07]	0.707	2

cosine: only the angle matters; lengths are divided out larger is better (1 = same direction)

cosine(q, A) cosine(q, B), 4 decimals first by cosine (A or B) cosine after doubling B

To do: Cosine of q and A (identical vectors)?
To do: Cosine of q = [1, 0] and B = [10, 10] (4 decimals)? dot 10, |q| = 1, |B| = 14.142
To do: Which document ranks first by cosine? Type A or B
To do: Doubling document B's vector to [20, 20] changes its cosine with q to…

Part 2. Do it in code

normalize(v): v divided by its length. ValueError for a zero vector.
cosine(a, b): dot(a, b) / (|a||b|). ValueError if either has length zero.
rank_by_cosine(q, docs): indices sorted by cosine, best first (ties: lower index first).
normalized_dot_equals_cosine(a, b): True if dot(normalize(a), normalize(b)) equals cosine(a, b) within 1e-12.
normalize_rows(docs): every row scaled to unit length (a zero row raises ValueError).

import numpy as np

def normalize(v):
    pass

def cosine(a, b):
    pass

def rank_by_cosine(q, docs):
    pass

def normalized_dot_equals_cosine(a, b):
    pass

def normalize_rows(docs):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

What is the cosine of two identical nonzero vectors?

ATwo, because both vectors contribute.BZero, because identical vectors cancel.COne: identical directions have an angle of 0, whatever their length.

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

\cos(q,x)=\frac{q^Tx}{\|q\|\|x\|},\quad \|q-x\|^2=\|q\|^2+\|x\|^2-2q^Tx.

For unit-normalized vectors, squared L2 is $2-2q^Tx$ , so their rankings agree under the appropriate direction. Zero vectors require a defined handling policy.

Worked example

Let q=[1,0], A=[1,0] and B=[10,10]. Raw dot products are 1 and 10, so B ranks first by dot product. Cosines are 1 and 1/sqrt(2), so A ranks first by cosine. This is a deliberate example of magnitude changing the retrieval objective.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import sqrt,isclose
q,a,b=[1,0],[1,0],[10,10]
dot=lambda u,v: sum(x*y for x,y in zip(u,v))
norm=lambda u: sqrt(dot(u,u))
assert dot(q,b) > dot(q,a)
assert dot(q,a)/(norm(q)*norm(a)) > dot(q,b)/(norm(q)*norm(b))
assert isclose(dot(q,b)/(norm(q)*norm(b)),1/sqrt(2))
assert 128*4 == 512

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Do not mix embedding spaces or metric directions. Normalize only when it matches the model and search objective. Similarity is not factual verification.