SD8.6 · LESSON 6 OF 6

Backpressure and poison jobs

Design asynchronous processing with explicit acknowledgment, ordering, replay and duplicate-handling rules.

THE IDEA

Backpressure and poison jobs

If jobs arrive faster than they are processed, the backlog grows by the difference: 120 in, 100 out gives 20 more every second, 1,200 after a minute; with arrivals back to 80 it takes another 60 seconds to drain. An unbounded queue just hides that, while delay and memory climb. Bound the queue, the retries and the time per job. A poison job that always fails must not be retried forever: after a few attempts move it to a dead-letter queue where someone can look at it.

arrivals (jobs/s)120processing (jobs/s)100seconds60max attempts per job3

backlog after 60 s: max(0, (120 − 100) × 60) = 1200 if arrivals fall to 80/s the backlog drains at 20/s: 60 s to clear unbounded growth hides the overload: delay and memory rise with the queue; a bounded queue rejects early so callers can back off a poison job that always fails is tried 3 time(s), then moved to a dead-letter queue for diagnosis if every job failed, handler calls per second could reach 120 × 3 = 360

Bound the queue, the retries and the processing time. A dead-letter path keeps failed work for diagnosis instead of retrying it forever.

Change arrivals, processing and retry limits.

Predict first. Does a queue with unlimited growth solve an overloaded consumer?

Chapter context

Asynchronous work decouples when a request is accepted from when its side effects finish. Queues and logs buffer differences in timing, but they do not remove processing cost or guarantee globally unique effects by themselves.

TRY IT YOURSELF

Part 1. Do it hands-on

Compute the backlog and drain time by hand, then write bounded queues and dead-lettering.

arrivals (jobs/s)120processing (jobs/s)100seconds60max attempts per job3

Bound the queue, the retries and the processing time. A dead-letter path keeps failed work for diagnosis instead of retrying it forever.

backlog after 60 s at 120 vs 100 seconds to clear at 80/s tries of a poison job (limit 3) unlimited queue solves overload? (yes/no)

To do: Jobs arrive at 120/s and are processed at 100/s. Backlog after one minute?
To do: Arrivals then drop to 80/s. Seconds to clear the 1200-job backlog (drain rate 100 − 80)?
To do: A poison job fails every time. With a limit of 3 attempts before it goes to the dead-letter queue, how many times is it tried?
To do: Does a queue with unlimited growth solve an overloaded consumer? Type yes or no

Part 2. Do it in code

backlog_after(arrival, service, seconds, start=0): simulate second by second: backlog = max(0, backlog + arrival − service).
time_to_clear(backlog, arrival, service): backlog / (service − arrival) seconds; infinity if service <= arrival.
bounded_queue(arrivals, capacity, service_per_step): arrivals is a list of jobs arriving each step. Each step: accept arrivals up to the free space (the rest are rejected), then process up to service_per_step jobs. Return {"accepted", "rejected", "left"} totals.
process_with_dlq(jobs, handler, max_attempts): for each job call handler(job) up to max_attempts times (True means success, False or an exception means failure). Return {"done": [...], "dead": [...], "attempts": {job: n}}.
retry_budget(jobs_per_s, max_attempts): the worst-case handler calls per second if every job fails: jobs_per_s × max_attempts.

def backlog_after(arrival, service, seconds, start=0):
    pass

def time_to_clear(backlog, arrival, service):
    pass

def bounded_queue(arrivals, capacity, service_per_step):
    pass

def process_with_dlq(jobs, handler, max_attempts):
    pass

def retry_budget(jobs_per_s, max_attempts):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Does a queue with unlimited growth solve an overloaded consumer?

AYesBNo

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

A fluid approximation for backlog B is

\Delta B\approx(\lambda-\mu)\Delta t

while the queue is nonempty, where lambda is arrival rate and mu processing rate. Backlog cannot go below zero. Delivery guarantees must specify which system boundary and side effects they cover.

Worked example

Arrivals at 120 jobs/s and processing at 100 jobs/s grow the queue by 20 jobs/s, reaching 1200 after one minute. If arrivals then drop to 80 jobs/s, net drain rate is 20 jobs/s and clearing that backlog takes 60 seconds under the constant-rate model.

Reference code

The following example uses the Python standard library and does not call a remote service.

arrival,service,seconds=120,100,60
backlog=max(0,(arrival-service)*seconds)
assert backlog == 1200
new_arrival=80
drain=service-new_arrival
assert drain == 20 and backlog/drain == 60

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Exactly-once terminology must state its scope. Retrying can duplicate external effects. A partition's order is not a universal order across an entire system.