R1.5 · LESSON 5 OF 6

Dimensions and storage

Rank documents by dot product, cosine and L2 distance and explain the role of normalization and model choice.

THE IDEA

Dimensions and storage

Every extra dimension costs space and arithmetic. A 128-dimensional float32 vector is 512 bytes. One million 768-dimensional vectors need 3.072 GB for the raw numbers alone, and every comparison does 768 multiplications. Doubling the dimension doubles both. Higher dimension does not automatically mean better retrieval: it has to be checked on your data.

dimensions768vectors1,000,000bytes per number4

one vector: 768 × 4 = 3,072 bytes all vectors: 3,072,000,000 bytes = 3.072 GB (before metadata and any index structure) one comparison: 768 multiplications

Storage and arithmetic grow linearly with the dimension. A bigger model is only worth it if it retrieves better on your data; measure that, do not assume it.

Change the dimension, the number of vectors and the precision.

Predict first. How many bytes are in a 128-dimensional float32 vector, excluding metadata?

Chapter context

Semantic search represents queries and documents as vectors in a compatible space. Good retrieval depends on the encoder, its training objective and the candidate data; choosing a distance formula alone does not create semantic relevance.

TRY IT YOURSELF

Part 1. Do it hands-on

Size a vector index by hand, then write the storage and cost helpers.

dimensions768vectors1,000,000bytes per number4

one vector: 768 × 4 = 3,072 bytes all vectors: 3,072,000,000 bytes = 3.072 GB (before metadata and any index structure) one comparison: 768 multiplications

Storage and arithmetic grow linearly with the dimension. A bigger model is only worth it if it retrieves better on your data; measure that, do not assume it.

bytes of a 128-d float32 vector GB for 10⁶ × 768-d float32 arithmetic ratio 1536-d vs 768-d

To do: Bytes in one 128-dimensional float32 vector (4 bytes per number), excluding metadata?
To do: One million 768-dimensional float32 vectors: decimal GB (3 decimals)?
To do: A 1536-dimensional model against a 768-dimensional one: how many times more arithmetic per comparison?

Part 2. Do it in code

bytes_per_vector(dim, bytes_per_number=4): dim · bytes_per_number.
index_bytes(n_vectors, dim, bytes_per_number=4): total raw vector storage (excluding metadata).
to_gb(n_bytes): decimal gigabytes.
mults_per_comparison(dim): the multiplications in one dot product (dim).
relative_cost(dim_a, dim_b): how many times more arithmetic per comparison dim_a needs than dim_b.
fits(n_vectors, dim, bytes_per_number, memory_bytes): True if the raw vectors fit in the memory budget.

def bytes_per_vector(dim, bytes_per_number=4):
    pass

def index_bytes(n_vectors, dim, bytes_per_number=4):
    pass

def to_gb(n_bytes):
    pass

def mults_per_comparison(dim):
    pass

def relative_cost(dim_a, dim_b):
    pass

def fits(n_vectors, dim, bytes_per_number, memory_bytes):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

How many float32 bytes are in a 128D vector, excluding metadata?

A1024, eight bytes per dimension.B512: 128 numbers × 4 bytes each.C128, one byte per dimension.

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

\cos(q,x)=\frac{q^Tx}{\|q\|\|x\|},\quad \|q-x\|^2=\|q\|^2+\|x\|^2-2q^Tx.

For unit-normalized vectors, squared L2 is $2-2q^Tx$ , so their rankings agree under the appropriate direction. Zero vectors require a defined handling policy.

Worked example

Let q=[1,0], A=[1,0] and B=[10,10]. Raw dot products are 1 and 10, so B ranks first by dot product. Cosines are 1 and 1/sqrt(2), so A ranks first by cosine. This is a deliberate example of magnitude changing the retrieval objective.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import sqrt,isclose
q,a,b=[1,0],[1,0],[10,10]
dot=lambda u,v: sum(x*y for x,y in zip(u,v))
norm=lambda u: sqrt(dot(u,u))
assert dot(q,b) > dot(q,a)
assert dot(q,a)/(norm(q)*norm(a)) > dot(q,b)/(norm(q)*norm(b))
assert isclose(dot(q,b)/(norm(q)*norm(b)),1/sqrt(2))
assert 128*4 == 512

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Do not mix embedding spaces or metric directions. Normalize only when it matches the model and search objective. Similarity is not factual verification.