A2.1 · LESSON 1 OF 6

State and actions

Design a bounded observe-act loop with explicit state, stopping conditions and a verifiable task outcome.

THE IDEA

State and actions

An agent is a loop: observe, choose an action, act, observe again. To design it, say precisely what it can observe and which actions it may request. In the room example the agent sees only its current room and, after inspecting, the items there. The key in room C is hidden state: nothing in A or B reveals it. And every observation has an age: if the world changes after it was taken, the agent is reasoning from a stale picture.

observation: { room: A, exits: [B], items: unknown (not inspected) } inventory: [] legal actions: move:B, inspect

The agent only knows what its observations show. The key is in room C, but nothing in A or B says so, and an observation can go stale: if the world changes after it was taken, the agent is still acting on the old picture. An external write counts as done only after a confirmation; a timeout means "unknown".

Move through the rooms. What do you actually know at each point?

Predict first. Can an agent safely assume an external write succeeded before receiving confirmation?

Chapter context

An agent chooses actions based on a task and observations, often over several model calls. Useful autonomy requires bounded resources, explicit feedback and a stopping rule that recognizes both success and inability to continue.

TRY IT YOURSELF

Part 1. Do it hands-on

Count actions and staleness by hand, then build the partially observable world.

observation: { room: A, exits: [B], items: unknown (not inspected) } inventory: [] legal actions: move:B, inspect

fewest moves A to C legal actions in B age of the step-1 observation at step 4 assume an unconfirmed write succeeded? (yes/no)

To do: Three rooms in a line: A – B – C. The agent starts in A and the key is in C. Fewest move actions to reach C?
To do: In room B the agent can move to each neighbouring room and inspect. How many actions are legal there?
To do: At step 1 the agent observed door = locked. At step 4 it still relies on that observation, though at step 3 someone unlocked the door. How many steps old is the observation at step 4?
To do: Can an agent safely assume an external write succeeded before it receives confirmation? Type yes or no

Part 2. Do it in code

The world (PROVIDED): GRAPH of rooms A – B – C in a line, ITEMS = {"C": ["key"]}, and a world dict {"room", "inspected", "inventory"} from new_world(). Items in a room are only visible AFTER inspecting that room (partial observability).
legal_actions(world): moves to each neighbouring room ("move:B", sorted), then "inspect", then "take:<item>" for each uninspected-and-visible item not already in the inventory (only if the current room has been inspected), sorted.
observe(world): {"room": ..., "exits": sorted neighbours, "items": sorted items still here if the room has been inspected, else None}.
step(world, action): a NEW world after the action (the input is not modified); an action that is not legal raises ValueError. inspect adds the room to inspected; take adds the item to the inventory.
shortest_moves(start, goal): the fewest moves between two rooms (BFS).
staleness(observed_at, now): how many steps old an observation is; ValueError if observed_at is after now.
write_status(result): "confirmed" if result has status "ok" and an id; "failed" if it has an "error"; otherwise "unknown" (a timeout is not a failure and not a success).

import copy

GRAPH = {"A": ["B"], "B": ["A", "C"], "C": ["B"]}
ITEMS = {"C": ["key"]}

def new_world():
    return {"room": "A", "inspected": [], "inventory": []}

def legal_actions(world):
    pass

def observe(world):
    pass

def step(world, action):
    pass

def shortest_moves(start, goal):
    pass

def staleness(observed_at, now):
    pass

def write_status(result):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Can an agent safely assume an external write succeeded before receiving confirmation?

ANoBYes

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

Represent a run as states s0,s1,... and actions a0,a1,... with observed transitions. A budget constraint can bound the sum of measured call costs. History-dependent token growth means a constant per-step estimate is only a simplifying model.

Worked example

A simulated agent may take at most three actions. It moves from room A to B, inspects a clue, then moves to C. The trace contains three actions. If the objective still is not satisfied, the correct result is budget exhausted with partial progress, not a fourth unapproved action or a fabricated success.

Reference code

The following example uses the Python standard library and does not call a remote service.

actions=['move:A->B','inspect:clue','move:B->C']
budget=3
assert len(actions) == budget
objective_satisfied=False
status='complete' if objective_satisfied else 'budget-exhausted'
assert status == 'budget-exhausted'
assert len(actions)*.002 == .006

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

A plan is not execution evidence. Reflection can repeat the original mistake. A step budget is useful only if every executed action is counted consistently.