R5.4 · LESSON 4 OF 6

Sharding and replicas

Design a retrieval record and explain filtering, indexing, partitioning, updates and consistency.

THE IDEA

Sharding and replicas

Shards divide the data so each server holds a part and can search it in parallel; replicas duplicate a shard for availability and extra query capacity. The two do opposite things to storage: 30.72 GB over 4 shards is 7.68 GB per shard, but 3 copies of everything is 92.16 GB across the system. A query goes to every shard, and the partial top-k lists are merged: this only works when scores are comparable across shards.

shards (split the data)4copies of each shard (replicas)1

shard	ids of 40 on it	data per shard server (GB)
0	10	7.68
1	10	7.68
2	10	7.68
3	10	7.68

data in the whole system: 30.72 GB (replicas = 1: no duplication) queries served: 1 × per-replica capacity (availability and throughput)

merge: shard 1 returns a1:0.9, a2:0.5; shard 2 returns b1:0.8, b2:0.7 global top-2 = a1, b1 (valid only if the scores are comparable across shards)

Change shards and replicas.

Predict first. Does adding a replica reduce the amount of original data stored across the whole system?

Chapter context

A vector index solves similarity lookup; a retrieval application also needs durable records, metadata, permissions and updates. Database behavior determines which vectors are eligible and which version of a document an answer uses.

TRY IT YOURSELF

Part 1. Do it hands-on

Size shards and replicas by hand, then write routing, merging and capacity helpers.

shards (split the data)4copies of each shard (replicas)1

shard	ids of 40 on it	data per shard server (GB)
0	10	7.68
1	10	7.68
2	10	7.68
3	10	7.68

data in the whole system: 30.72 GB (replicas = 1: no duplication) queries served: 1 × per-replica capacity (availability and throughput)

merge: shard 1 returns a1:0.9, a2:0.5; shard 2 returns b1:0.8, b2:0.7 global top-2 = a1, b1 (valid only if the scores are comparable across shards)

GB per shard, 4 shards GB total with 3 copies merged top-2 ids replica shrinks stored data? (yes/no)

To do: 30.72 GB of vectors split evenly across 4 shards (no replicas): GB per shard?
To do: The same data with 3 copies of every shard: total GB stored across the system?
To do: Shard 1 returns (a1, 0.9), (a2, 0.5); shard 2 returns (b1, 0.8), (b2, 0.7); higher score is better. The global top-2 ids, comma separated?
To do: Does adding a replica reduce the amount of original data stored across the whole system? Type yes or no

Part 2. Do it in code

shard_of(doc_id, n_shards): a deterministic shard: the sum of the character codes (ord) of the id modulo n_shards. n_shards < 1 raises ValueError.
balance(ids, n_shards): the list of how many ids land on each shard.
merge_topk(results, k, higher_is_better=True): results is a list (one per shard) of (id, score) pairs; return the k best (id, score) pairs overall (ties: smaller id).
per_shard_gb(total_gb, n_shards, replicas=1): data stored on each shard server: total_gb / n_shards · replicas.
total_gb(total_gb, replicas): data stored across the whole system.
capacity_qps(replicas, qps_per_replica): queries per second the replicas can serve together.

def shard_of(doc_id, n_shards):
    pass

def balance(ids, n_shards):
    pass

def merge_topk(results, k, higher_is_better=True):
    pass

def per_shard_gb(total_gb, n_shards, replicas=1):
    pass

def total_gb(total_gb, replicas):
    pass

def capacity_qps(replicas, qps_per_replica):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Does adding a replica reduce the amount of original data stored across the whole system?

ANoBYes

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

Raw vector storage is $M=Ndb$ for N vectors of dimension d and b bytes per coordinate. If a postfilter independently accepts fraction f of an initial k candidates, the expected survivor count is kf; this is an expectation, not a guarantee.

Worked example

Ten million vectors of dimension 768 in float32 require 10,000,000×768×4=30,720,000,000 bytes, or 30.72 decimal GB, before IDs, text, indexes and replicas. If a postfilter accepts 20% of ten candidates under a simplifying independence assumption, the expected survivor count is only two.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import isclose
bytes_used=10_000_000*768*4
assert bytes_used == 30_720_000_000
assert isclose(bytes_used/1_000_000_000,30.72)
assert 10*.2 == 2

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Filter selectivity and ANN candidate recall interact. Replication adds storage. Matching vector dimension does not establish matching embedding space.