R4.6 · LESSON 6 OF 6

Quality and memory budgets

Calculate vector-code storage and distinguish scalar, product and binary quantization from reranking.

THE IDEA

Quality and memory budgets

Honest comparisons state a storage budget and what it includes: codes, ids, graph edges, codebooks, any original vectors kept for reranking, and alignment padding. For a million vectors, PQ codes are 96 MB but the full resident set can be 232.8 MB with ids, a graph and codebooks, and 3.3 GB if the float32 originals stay in memory. Then the compression headline almost disappears.

component (1,000,000 vectors)	bytes	share
PQ codes (96 B each)	96,000,000
ids (8 B each)	8,000,000
graph edges (32 × 4 B per node)	128,000,000
codebooks (shared)	786,432
original float32 vectors	3,072,000,000

total 3,304,786,432 bytes = 3.305 GB (raw float32 vectors alone: 3.072 GB) code bytes alone are 2.9% of what is resident

Compare quality at a stated storage budget and state exactly what is counted: codes, ids, graph edges, codebooks, originals and any alignment padding.

Switch components on and off.

Predict first. If originals remain resident, does reporting only code bytes describe total storage?

Chapter context

Compressed vectors reduce storage and memory bandwidth, but approximate distance calculations can change rankings. Separate the compressed search representation from any original vectors retained for a later accurate rerank.

TRY IT YOURSELF

Part 1. Do it hands-on

Total a memory budget by hand, then write the breakdown and budget checks.

component (1,000,000 vectors)	bytes	share
PQ codes (96 B each)	96,000,000
ids (8 B each)	8,000,000
graph edges (32 × 4 B per node)	128,000,000
codebooks (shared)	786,432
original float32 vectors	3,072,000,000

total 3,304,786,432 bytes = 3.305 GB (raw float32 vectors alone: 3.072 GB) code bytes alone are 2.9% of what is resident

Compare quality at a stated storage budget and state exactly what is counted: codes, ids, graph edges, codebooks, originals and any alignment padding.

total bytes without originals GB with originals code share of that total code bytes alone = total? (yes/no)

To do: 1,000,000 vectors: PQ codes 96 B each, ids 8 B each, a graph with 32 neighbours × 4 B per node, and 786,432 B of codebooks. Total bytes without the original vectors?
To do: If the 3,072,000,000 bytes of float32 originals are also kept for reranking, total decimal GB (3 decimals)?
To do: Then the compressed codes alone are what fraction of that total (3 decimals)?
To do: If originals stay resident, does reporting only the code bytes describe the total storage? Type yes or no

Part 2. Do it in code

breakdown(n, dim, code_bytes, id_bytes, edges_per_node, edge_bytes, codebook_bytes, keep_originals, float_bytes=4): a dict with codes, ids, graph, codebooks, originals (0 when not kept) and total.
to_gb(n_bytes): decimal gigabytes.
code_share(b): codes / total for a breakdown dict.
ratio_vs_raw(b, n, dim, float_bytes=4): raw float32 size divided by the total of the breakdown.
within_budget(b, budget_bytes): True if the total is at most the budget.

def breakdown(n, dim, code_bytes, id_bytes, edges_per_node, edge_bytes, codebook_bytes, keep_originals, float_bytes=4):
    pass

def to_gb(n_bytes):
    pass

def code_share(b):
    pass

def ratio_vs_raw(b, n, dim, float_bytes=4):
    pass

def within_budget(b, budget_bytes):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

If originals remain resident, does reporting only code bytes describe total storage?

ANoBYes

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

For d float32 coordinates, raw vector bytes are 4d. For M PQ subspaces and b bits per subspace, code bytes are $Mb/8$ when packed. For equal-size subspaces, float32 codebooks total $2^b d\times4$ bytes, excluding additional index data.

Worked example

A 768D float32 vector uses 768×4=3072 bytes. With 96 subspaces and 8-bit codes it uses 96 bytes: 32 times smaller for the per-vector representation. The shared codebooks contain 256×768 float32 values, requiring 786,432 bytes. Original vectors retained for reranking are an additional cost.

Reference code

The following example uses the Python standard library and does not call a remote service.

dimension,parts,bits=768,96,8
raw=dimension*4
code=parts*bits//8
book=(2**bits)*dimension*4
assert raw == 3072 and code == 96
assert raw/code == 32
assert book == 786432
assert sum(a!=b for a,b in zip('1010','1111')) == 2

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Compression ratio for code bytes is not the total-system ratio. PQ codes are indices into learned representatives, not simply truncated float bits. Reranking cannot find missing candidates.