R3.6 · LESSON 6 OF 6

Tuning and updates

Trace IVF and HNSW search and explain how candidate exploration trades latency for recall.

THE IDEA

Tuning and updates

Tuning means choosing a point on the recall–latency–memory curve that meets your requirement on representative data. In the toy table the smallest efSearch with recall ≥ 0.97 within 10 ms is 64 (9 ms). Indexes also age: heavy inserts and deletes erode quality, and a new embedding model changes the space, so the index must be re-embedded, rebuilt and re-measured.

minimum recall0.97maximum latency (ms)10

efSearch	recall	latency (ms)	meets both?
16	0.9	3	no
32	0.95	5	no
64	0.98	9	yes
128	0.99	17	no

chosen setting: ef = 64 (9 ms → 111.1 queries/s per worker)

A value that suits one corpus is not a universal recommendation: measure on representative data. Re-measure after major changes, and rebuild if the embedding model changes, because new vectors live in a different space.

Set the requirement, read off the setting.

Predict first. Should index quality be rechecked after a major embedding-model change?

Chapter context

An approximate index organizes vectors so a query can avoid most comparisons. Clustering and graph navigation use different mechanisms to choose candidates, and their build settings influence later search quality.

TRY IT YOURSELF

Part 1. Do it hands-on

Pick a setting by hand, then write the tuning and rebuild-decision helpers.

minimum recall0.97maximum latency (ms)10

efSearch	recall	latency (ms)	meets both?
16	0.9	3	no
32	0.95	5	no
64	0.98	9	yes
128	0.99	17	no

chosen setting: ef = 64 (9 ms → 111.1 queries/s per worker)

smallest ef (recall ≥ 0.97, ≤ 10 ms) its latency (ms) queries per second per worker recheck after a model change? (yes/no)

To do: Measured curve: ef 16 → recall 0.90, 3 ms; ef 32 → 0.95, 5 ms; ef 64 → 0.98, 9 ms; ef 128 → 0.99, 17 ms. The application needs recall ≥ 0.97 and at most 10 ms. Smallest ef that meets both?
To do: Its latency in ms?
To do: Queries per second for one worker at that latency (1 decimal)?
To do: You switch to a different embedding model. Must index quality be rechecked (and the vectors re-embedded)? Type yes or no

Part 2. Do it in code

pick_setting(curve, min_recall, max_ms): curve is a list of dicts {param, recall, ms} in ascending param order. Return the smallest param whose recall is at least min_recall AND ms is at most max_ms, or None.
qps(ms): queries per second for one worker (1000 / ms).
needs_rebuild(meta, model, dim): meta is {"model":..., "dim":...}; True if the model or the dimension differs.
should_rebuild(n_built, n_added, n_deleted, threshold): True if (added + deleted) / built is strictly greater than the threshold.
recall_drift(old, new, tol): True if the new recall is more than tol below the old one.

def pick_setting(curve, min_recall, max_ms):
    pass

def qps(ms):
    pass

def needs_rebuild(meta, model, dim):
    pass

def should_rebuild(n_built, n_added, n_deleted, threshold):
    pass

def recall_drift(old, new, tol):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Should index quality be rechecked after a major embedding-model change?

AYesBNo

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

Under the simplifying assumption of equally sized IVF lists, N vectors, C cells and p probes produce approximately

N_{candidates}\approx Np/C.

Add coarse scoring, uneven list sizes and any reranking costs separately. HNSW has no equivalent universal fixed candidate-count formula.

Worked example

For 102,400 vectors evenly divided among 1024 cells, a cell has 100 vectors. Probing eight cells scans about 800 candidates, compared with 102,400 in a full scan. This excludes coarse-centroid comparisons and says nothing by itself about recall.

Reference code

The following example uses the Python standard library and does not call a remote service.

total,cells,probes=102400,1024,8
assert total/cells == 100
assert total*probes/cells == 800
assert total/(total*probes/cells) == 128

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

An average candidate count is not an actual worst-case bound. More graph edges consume memory. Approximate search settings must be evaluated against the current corpus and metric.