SD2.4 · LESSON 4 OF 6

Pagination

Trace a client request and design an HTTP contract with clear failure, retry and pagination behavior.

THE IDEA

Pagination

Offset pagination ("skip 3, take 3") counts positions. If a new item is inserted at the top between two requests, every position shifts and page 2 repeats the last item of page 1. Cursor pagination ("give me items after id 8") names the last item seen, so it keeps its place, but it needs a stable ordering and a clear contract for what the next cursor means.

page 1 (both styles): 10, 9, 8 feed now: 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 page 2 by OFFSET 3: 8, 7, 6 ← repeats 8 page 2 by CURSOR after 8: 7, 6, 5 (no repeats)

Offsets count positions, and positions move when rows are inserted or deleted. A cursor names the last item seen, so it needs a stable ordering and a defined continuation rule.

Insert an item between the two page requests.

Predict first. Can inserting records change which items fall on an offset-based next page?

Chapter context

Clients communicate through name resolution, connections, transport security and application protocols. API semantics determine what a caller can safely infer or repeat when a network interruption hides an operation's outcome.

TRY IT YOURSELF

Part 1. Do it hands-on

Walk a changing feed by hand, then write both pagination styles and detect repeats.

page 1 (both styles): 10, 9, 8 feed now: 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 page 2 by OFFSET 3: 8, 7, 6 ← repeats 8 page 2 by CURSOR after 8: 7, 6, 5 (no repeats)

Offsets count positions, and positions move when rows are inserted or deleted. A cursor names the last item seen, so it needs a stable ordering and a defined continuation rule.

offset page 2 starts at id items repeated cursor page 2 starts at id inserts can shift offset pages? (yes/no)

To do: A feed lists newest first: ids 10, 9, 8, …, 1, page size 3. After the client reads page 1 (10, 9, 8), a new item 11 arrives at the top. Page 2 by OFFSET 3 starts with which id?
To do: How many items of page 2 were already seen on page 1?
To do: Page 2 by CURSOR "after id 8" (items with a smaller id) starts with which id?
To do: Can inserting records change which items fall on an offset-based next page? Type yes or no

Part 2. Do it in code

FEED = newest-first list of ids is passed in; page size is `limit`.
offset_page(items, offset, limit): items[offset: offset + limit].
cursor_page(items, after, limit): items are sorted descending by id. Return {"items": the next `limit` items with id < after (all items when after is None), "next": the last returned id, or None when fewer than `limit` items were returned}.
walk_offset(feed_at, limit, steps): feed_at(step) returns the current feed at that step. Read `steps` pages by offset (offset = limit × page number) and return the list of ids seen, in order.
walk_cursor(feed_at, limit, steps): the same, following the `next` cursor.
duplicates(seen): the sorted ids that appear more than once.

def offset_page(items, offset, limit):
    pass

def cursor_page(items, after, limit):
    pass

def walk_offset(feed_at, limit, steps):
    pass

def walk_cursor(feed_at, limit, steps):
    pass

def duplicates(seen):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Can inserting records change which items fall on an offset-based next page?

ANoBYes

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

For k sequential calls with round-trip latency r, the network component alone is approximately k*r under a simple fixed-RTT model. Independent parallel calls have an ideal max-path cost instead. This excludes connection setup, payload transfer, queueing and application computation.

Worked example

Three required sequential remote calls with 40 ms round trips consume about 120 ms of round-trip time before server work. For a separate write example, processing request key op-7 twice should create one logical result if the idempotency contract is implemented atomically. The calculation below is an offline model, not a distributed storage implementation.

Reference code

The following example uses the Python standard library and does not call a remote service.

assert 3*40 == 120
seen={}
def create(key,value):
    if key in seen:
        if seen[key] != value: raise ValueError('key reused with different intent')
        return seen[key]
    seen[key]=value
    return value
assert create('op-7','order-1') == create('op-7','order-1')
assert len(seen) == 1

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

A timeout is not proof of rollback. HTTP transport security does not authorize object access. An in-memory deduplication dictionary does not supply durability or atomicity across servers.