Normalization and read models
Choose representations and storage boundaries from access patterns, ownership and retention requirements.
THE IDEA
Normalization and read models
Normalisation stores each fact once, which avoids update anomalies. A denormalised read view copies facts into the records that need them so reads are cheap: a feed that embeds the author's name needs no author lookups. The price is that every copy can go stale: a rename leaves 10,000 old names until something rewrites them. So a denormalised view needs an explicit update path and a repair job that is safe to repeat.
Normalisation avoids duplicated facts and update anomalies. A denormalised read view makes queries cheaper but needs an explicit update and repair path, because every copy is another thing that can disagree with the source.
Predict first. If a user's name is copied into many feed records, can those copies become stale?
Chapter context
A data model describes the facts a system owns and the relationships it must preserve. Storage choices should follow how those facts are queried, updated and retained, rather than a blanket preference for one database category.
TRY IT YOURSELF
Part 1. Do it hands-on
Count stale copies and lookups by hand, then build both feeds and a repair.
Normalisation avoids duplicated facts and update anomalies. A denormalised read view makes queries cheaper but needs an explicit update and repair path, because every copy is another thing that can disagree with the source.
- To do: A user's name is copied into 10,000 feed records. The user renames; only the users table is updated. Stale copies?
- To do: A normalised feed page shows 20 posts from 8 distinct authors and fetches each distinct author once. Extra lookups per page?
- To do: A repair job rewrites all stale copies. Records rewritten?
- To do: If a user's name is copied into many feed records, can those copies become stale? Type yes or no
Part 2. Do it in code
- users = {uid: {"name"}}; posts = list of {"id", "author"}.
- build_feed(posts, users, denormalize): normalised feed → [{"id", "author"}] (the name is looked up when the feed is read, so build_feed returns the posts unchanged); denormalised feed → [{"id", "author", "author_name"}] with the name copied in now.
- read_feed(feed, users): what the reader sees: each item with an author_name — taken from the copy when present, otherwise looked up in users.
- lookups_needed(feed, users): the number of distinct authors that must be looked up for items without a copied name.
- stale_copies(feed, users): the ids of feed items whose copied author_name differs from the current name in users.
- repair(feed, users): refresh every stale copy; return how many were rewritten.
Stuck, or something not working? You can come back to it any time.
CHECK YOUR UNDERSTANDING
If a user's name is copied into many feed records, can those copies become stale?
Chapter reference: math, worked example & code
This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.
Math and notation
A simple retained-payload estimate is
where w is writes/day, s payload bytes/write, d retained days and r full-copy count. Add indexes, metadata, compaction headroom and backups separately; replication schemes may store more than exactly r payload copies.
Worked example
Two million records per day at 500 bytes each produce one decimal GB/day of raw payload. Thirty days retained across three full copies require 2,000,000×500×30×3=90,000,000,000 bytes, or 90 decimal GB before other storage. Compressibility and index size are separate measured factors.
Reference code
The following example uses the Python standard library and does not call a remote service.
daily,size,days,copies=2_000_000,500,30,3
assert daily*size == 1_000_000_000
retained=daily*size*days*copies
assert retained == 90_000_000_000
assert retained/1e9 == 90
Python examples are provided for study and local execution; this page does not execute Python.
Common mistakes
NoSQL is not a single consistency or query model. Denormalization creates a maintenance responsibility. Payload estimates must not silently stand in for provisioned disk requirements.