SD5.1 · LESSON 1 OF 6

Cache identity

Design a cache key, freshness policy and miss path while calculating load and mean-latency effects.

THE IDEA

Cache identity

A cache answers "have I stored the response to this request?" so the key must contain every dimension that changes the response: the resource, the version, the locale, and the authorisation scope. Keyed by URL alone, user B's request for /profile can be answered with user A's cached page. Add too many dimensions, though, and the cache splits into many rarely reused entries: 3 locales × 2 versions already need 6.

request	cache key	result	served body
A, en	/profile\|en	miss	profile of A (en)
B, en	/profile\|en	hit	profile of A (en) ← WRONG
A, fr	/profile\|fr	miss	profile of A (fr)

The key must include every dimension that changes the response: resource, version, locale, authorisation scope. Too little in the key serves the wrong response (or someone else's private data); too much splits the cache into entries that are rarely reused. Three locales × two versions need 6 entries.

Toggle what goes into the key and watch for wrong answers.

Predict first. Can two users always share a private response keyed only by URL?

Chapter context

A cache serves a previously obtained representation instead of repeating work. Its correctness depends on which requests can share a value and how long that value may remain useful. Fast retrieval of the wrong representation is still an error.

TRY IT YOURSELF

Part 1. Do it hands-on

Count entries and a leak by hand, then build keyed serving and a leak detector.

request	cache key	result	served body
A, en	/profile\|en	miss	profile of A (en)
B, en	/profile\|en	hit	profile of A (en) ← WRONG
A, fr	/profile\|fr	miss	profile of A (fr)

entries for 3 locales × 2 versions entries for 2 users (URL + user key) URL-only key: B receives whose data? (A/B) users can share private responses by URL? (yes/no)

To do: The home page varies by locale (en, fr, de) and by deployed version (v1, v2). Distinct cache entries needed to keep every combination apart?
To do: GET /profile returns the logged-in user's private data. Keyed by URL AND user, two users need how many entries?
To do: Keyed by URL only: user A requests /profile (miss, cached), then user B requests /profile. Whose data does B receive? Type A or B
To do: Can two users always share a private response keyed only by URL? Type yes or no

Part 2. Do it in code

make_key(req, vary): a tuple of the request's url followed by req[d] for each dimension d in vary (for example locale, version, user).
serve(requests, origin, vary): serve each request through a cache keyed by make_key. origin(req) returns the body on a miss. Return the list of (body, hit) pairs.
leaks(requests, origin, vary): the number of requests whose body was produced for a DIFFERENT user than the one asking (compare the user in the request with the user the body was generated for, via origin_user(body)); a correct key gives 0.
entries_needed(dimensions): the product of the number of values in each dimension (a dict name → list of values).
origin_user(body): the user name embedded in the body, which has the form "<url> for <user>".

def make_key(req, vary):
    pass

def serve(requests, origin, vary):
    pass

def origin_user(body):
    pass

def leaks(requests, origin, vary):
    pass

def entries_needed(dimensions):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Can two users always share a private response keyed only by URL?

ANoBYes

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

If h is hit probability, t-hit hit-path time and t-miss complete miss-path time:

E[T]=h\,t_{hit}+(1-h)t_{miss},\quad R_{origin}\approx(1-h)R.

The miss time here already includes lookup overhead. Means do not establish tail latency, and background refresh can add origin traffic.

Worked example

At 90% hits, 2-ms hit paths and 50-ms complete miss paths, mean latency is 0.9×2+0.1×50=6.8 ms. For 1000 requests/s, nominal origin demand is 100 requests/s. During a complete cache outage, it could become 1000 requests/s rather than remaining at the normal miss rate.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import isclose
hit=.9
assert isclose(hit*2+(1-hit)*50,6.8)
assert isclose((1-hit)*1000,100)
assert 1000/100 == 10

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

A hit ratio is not a correctness guarantee. Mean latency does not reveal p95. Shared caching requires a safe identity and privacy policy; TTL does not replace authorization.