SD1.6 · LESSON 6 OF 6

Constraints and decisions

Turn a product requirement into a workload model, measurable objectives and a justified initial capacity estimate.

THE IDEA

Constraints make the design

A design is a set of choices made under limits. The estimate tells you what the system needs. The constraints tell you what you are allowed to spend to get there: money, people, existing systems, deadlines. A diagram with six services, three databases and a message bus may be a fine design for a company with fifty engineers and a bad one for two.

Decision log entry (one line each is enough)

Decision:     Managed database with a standby, not a self-run sharded cluster
Because:      2 engineers; cluster costs 5 of our 5 ops points; one primary has 3.4x read runway with a cache
Next step:    when reads exceed ~1/3 of capacity, add read replicas before sharding
Revisit if:   write rate passes 50% of the primary's capacity

A good decision names the bottleneck it accepts and the next scaling step, so growth is a plan, not a surprise.

Runway: how far away is the first bottleneck?

Every resource has a capacity and the planning peak loads it. Capacity divided by load is its runway: how many times traffic could grow before it runs out. The smallest runway is the bottleneck. Try adding a cache and see which resource it helps.

feed cache (70% hit)App instances 5Database

App instances: 1.62x the planning peak

Database reads: 1.19x the planning peak (first to run out)

Database writes: 6.74x the planning peak

Runway is capacity divided by planning-peak load. The smallest one is the bottleneck and the next scaling step. Turn the cache on and watch the database reads bar jump while the app bar does not move.

Planning peak is 246.9 req/s. The app tier scales with instance count, the database is a fixed amount per option.

Predict first. You add a feed cache with a 70% hit rate. Which bar moves most?

The common mistake: copying the reference diagram

Reference architectures show every part a very large system might need. Adopting all of them in release one gives you the cost, the operational load and the failure modes of each, before any requirement asks for them. Start from the requirement, choose the simplest option that meets it with margin and say when you would change it.

Pick the least complex design that meets the objectives, state its first bottleneck and the next step.

Don't

Add services "for scale" without a number. In the practice you will see six services and a sharded cluster exceed both budget and team capacity.

Chapter context

System design starts with what the system must do and the conditions under which it must do it. A component diagram is only useful when its choices can be traced back to access patterns, data volume, latency targets and failure expectations.

TRY IT YOURSELF

The team drafted a design that looks impressive. Constraints: 4,000 dollars per month, two engineers (at most 5 ops points), mean latency within 50 ms at the planning peak even with one app instance down, survive one database host failure, and keep at least 1.5x runway to the first bottleneck. All prices and ops points are invented for this exercise. Revise the decisions until every check passes.

Architecture Database

CDN for images (600 dollars, 1 ops, serves 90% of image bytes) Managed feed cache (250 dollars, 1 ops, 70% hit)App instances (150 dollars each) 3

Monthly cost5,100

includes 10 Gbit/s origin link (2500) and 300 base

Ops load6.5 of 5

Latency, 1 app instance lostsaturated

Smallest runway0.97x

App instances: 0.97x the planning peak (first to run out)

Database reads: 1.19x the planning peak

Database writes: 6.74x the planning peak

First resource to run out

Checks (0 of 6)

Monthly cost is within the 4,000 dollar budget (illustrative prices)
Operational load fits the team of two (at most 5 ops points)
With one app instance lost at the planning peak, mean latency stays within 50 ms
A single database host failure does not take the service down
The first bottleneck is at least 1.5 times away from the planning peak
Name the resource that would run out first as traffic grows

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Must a first release use every distributed component in a reference diagram?

ANoBYes

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

For U daily active users and a requests per user per day:

R_{avg}=Ua/86400,\quad R_{peak}=pR_{avg},\quad n=\lceil R_{peak}/(cu)\rceil.

Here p is a peak multiplier, c measured capacity per instance and u a chosen usable fraction. The multiplier and capacity are workload assumptions, not universal constants.

Worked example

Assume 100,000 daily users, 20 requests each and a peak multiplier of 10. Average demand is 2,000,000/86,400≈23.15 requests/s and estimated peak is 231.48 requests/s. An instance sustaining 100 requests/s with usable fraction 0.6 contributes 60 requests/s, so four instances cover the estimated peak. Five are needed to retain that same usable capacity after losing one instance.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import ceil,isclose
average=100_000*20/86400
peak=10*average
usable=100*.6
assert isclose(average,23.14814814814815)
assert ceil(peak/usable) == 4
assert (5-1)*usable >= peak
assert (4-1)*usable < peak

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Average traffic is not peak traffic. A headroom assumption is not measured performance. Durability, correctness and availability need separate objectives.