SD6.6 · LESSON 6 OF 6

Failure capacity

Route requests across instances and calculate capacity with headroom, startup time and instance failures.

THE IDEA

Failure capacity

With demand 900 req/s and instances that sustain 400 but are run at a 75% target (300 req/s each), three instances meet demand. To keep the same target after losing one you need four: (4 − 1) × 300 = 900. Headroom for normal load is not failure capacity, and the plan must also respect the limits of what you depend on: 4 instances × 50 connections is exactly the 200 the database allows; a fifth would not fit. When an instance disappears, its traffic and connections shift onto the rest abruptly.

demand (req/s)900target usage0.75failures to survive1database connections per instance50

each instance: 400 req/s, usable 300 at the target needed normally: ceil(900 / 300) = 3 surviving 1 failure(s): 4 instances → 3 left carry 900 req/s database allows 200 connections → at most 4 instances at 50 each → fits

Steady-state headroom is not failure capacity. Reserve enough for the failure you intend to survive, check the limits of the systems you depend on, and remember that losing an instance moves traffic and connections abruptly onto the rest.

Change demand, target usage and the failures to survive.

Predict first. Is steady-state headroom automatically sufficient for every node-loss scenario?

Chapter context

Load balancing distributes work among eligible backends. Scaling changes available capacity. Both require the application to manage state and dependencies so adding a process actually adds useful throughput.

TRY IT YOURSELF

Part 1. Do it hands-on

Size a fleet for a failure by hand, then write the planner and the dependency check.

demand (req/s)900target usage0.75failures to survive1database connections per instance50

instances needed (900, 400, 75%) to survive one loss req/s carried by 3 of 4 instances the DB supports steady headroom covers any node loss? (yes/no)

To do: Demand 900 requests/s; each instance can do 400 but the target is 75% usage (300 req/s). Instances needed normally?
To do: To keep the same target after losing one instance?
To do: Capacity at the 75% target with 4 instances after one fails, in requests/s?
To do: Each instance opens 50 database connections; the database allows 200. Most instances the database can support?
To do: Is steady-state headroom automatically sufficient for every node-loss scenario? Type yes or no

Part 2. Do it in code

instances_needed(demand, capacity, usable): ceil(demand / (capacity × usable)).
capacity_after_failures(n, capacity, usable, failed): (n − failed) × capacity × usable (not below 0).
instances_with_failures(demand, capacity, usable, failures): the smallest n such that capacity_after_failures(n, …, failures) >= demand.
max_instances_by_limit(dependency_limit, per_instance): the most instances the dependency can support (limit // per_instance).
plan(demand, capacity, usable, failures, dependency_limit, per_instance): returns {"instances": needed with failures, "max_supported": ..., "ok": instances <= max_supported}.

import math

def instances_needed(demand, capacity, usable):
    pass

def capacity_after_failures(n, capacity, usable, failed):
    pass

def instances_with_failures(demand, capacity, usable, failures):
    pass

def max_instances_by_limit(dependency_limit, per_instance):
    pass

def plan(demand, capacity, usable, failures, dependency_limit, per_instance):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Is steady-state headroom automatically sufficient for every node-loss scenario?

AYesBNo

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

With demand R, per-instance capacity c and intended usable fraction u, require $ncu\ge R$ . To keep that same target after one failure, require $(n-1)cu\ge R$ . These are rate constraints, not a complete queueing or availability model.

Worked example

Demand is 900 requests/s. Each instance sustains 400 requests/s, but the target usable fraction is 75%, or 300 requests/s. Three instances meet normal demand. Four are required to maintain the same target capacity after losing one: (4−1)×300=900.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import ceil
demand,capacity,usable=900,400,.75
normal=ceil(demand/(capacity*usable))
resilient=normal+1
assert normal == 3 and resilient == 4
assert (resilient-1)*capacity*usable == demand

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Horizontal scaling can move the bottleneck to a database. Healthy and ready are not identical. A load balancer does not repair session or transaction state lost with a backend.