SD11.4 · LESSON 4 OF 6

Validation and resource use

Place authentication, object authorization and resource limits at explicit application boundaries.

THE IDEA

Validation and resource use

Validate shape and cost. Bound the body size, the nesting depth, the number of items, the page size and the expansion of nested lists, because a syntactically valid request can still exhaust resources: three nested lists of 100 items make a million. Clamp page sizes on the server and reject over-complex queries early.

nesting of the payload3requested page size10000nested list 1 size100nested list 2 size100nested list 3 size100

payload depth 3 → accepted page size 10000 → clamped to 100 (maximum 100) expansion 100 × 100 × 100 = 1,000,000 items → REJECT: over the 10,000-item complexity budget the request can be perfectly valid JSON and still be expensive

Change the nesting, page size and list sizes.

Predict first. Can a valid deeply nested payload need a complexity limit?

Chapter context

A distributed application must identify callers and control which resources and effects they may access. Tenant boundaries and resource budgets must be enforced by trusted components rather than inferred from client-supplied identifiers.

TRY IT YOURSELF

Part 1. Do it hands-on

Measure depth and expansion by hand, then write the limit checks.

nesting of the payload3requested page size10000nested list 1 size100nested list 2 size100nested list 3 size100

depth of {a:{b:{c:1}}} page size 10,000 → 100 × 100 × 100 items valid nested payloads need a limit? (yes/no)

To do: The JSON {"a": {"b": {"c": 1}}} has nesting depth (each object level counts 1)?
To do: A client asks for page_size = 10,000 and the server maximum is 100. Items returned per page?
To do: A query expands 3 nested lists of 100 items each (100 × 100 × 100). Items the server would build?
To do: Can a syntactically valid, deeply nested payload need a complexity limit? Type yes or no

Part 2. Do it in code

depth(obj): 0 for a scalar; 1 + the deepest child for a dict (its values) or a list (its items). An empty container has depth 1.
count_items(obj): the total number of values in the structure (scalars count 1; containers count their contents recursively; an empty container counts 0).
check_payload(obj, size_bytes, limits): limits has max_bytes, max_depth, max_items. Return a list of violations in this order: "too_large", "too_deep", "too_many_items" (empty list if fine).
clamp_page_size(requested, default, maximum): default if requested is None, otherwise min(max(requested, 1), maximum).
expansion(list_sizes): the product of the nested list sizes (items the server would build).
within_budget(list_sizes, max_items): expansion(list_sizes) <= max_items.

def depth(obj):
    pass

def count_items(obj):
    pass

def check_payload(obj, size_bytes, limits):
    pass

def clamp_page_size(requested, default, maximum):
    pass

def expansion(list_sizes):
    pass

def within_budget(list_sizes, max_items):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Can a valid deeply nested payload need a complexity limit?

ANoBYes

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

For a token bucket with capacity C, refill rate r, previous tokens b and elapsed time dt:

b'=\min(C,b+r\,dt).

Accept cost k only when $b'\ge k$ , then subtract k atomically. Clock behavior, concurrency and shared enforcement are additional implementation requirements.

Worked example

A bucket holds at most 10 tokens and refills 2/s. An 8-token request at time zero leaves 2. Two seconds later it has 6. A request costing 7 is denied and leaves 6; a request costing 6 is accepted and leaves zero. Rejected operations should not accidentally create negative credit under this contract.

Reference code

The following example uses the Python standard library and does not call a remote service.

capacity,rate,tokens=10,2,10
tokens-=8
assert tokens == 2
tokens=min(capacity,tokens+rate*2)
assert tokens == 6
assert tokens < 7
assert tokens >= 6
tokens-=6
assert tokens == 0

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Authentication is not object-level authorization. Cache and worker paths can cross tenant boundaries too. A global rate limit requires a stated distributed enforcement contract.