Reflection and evaluation
Design a bounded observe-act loop with explicit state, stopping conditions and a verifiable task outcome.
THE IDEA
Reflection and evaluation
A reflection step asks the agent (or another model) to review its work and spot gaps. That helps, but a review is another model judgement, not ground truth. Grade an agent on what actually happened: the state of the world and the effects of its tool calls. In a sample of ten runs where the agent claimed success nine times but only six really succeeded, its self-reported rate (0.9) overstates the true rate (0.6) by 0.3.
A reflection step can point out gaps in the evidence, but another model's judgement is not ground truth either. Measure real task success and real tool effects.
Predict first. Is an agent's self-reported success sufficient for grading a task?
Chapter context
An agent chooses actions based on a task and observations, often over several model calls. Useful autonomy requires bounded resources, explicit feedback and a stopping rule that recognizes both success and inability to continue.
TRY IT YOURSELF
Part 1. Do it hands-on
Compare claimed and true success by hand, then write the evaluation helpers.
A reflection step can point out gaps in the evidence, but another model's judgement is not ground truth either. Measure real task success and real tool effects.
- To do: Ten graded runs. The agent claimed success in 9 of them. Self-reported success rate?
- To do: Checking the actual state, 6 runs really succeeded. True success rate?
- To do: How many runs were claimed successful but did not actually succeed?
- To do: Is an agent's self-reported success sufficient for grading a task? Type yes or no
Part 2. Do it in code
- rates(runs): runs is a list of {"claimed": bool, "ok": bool} (ok = the real outcome checked from state). Return {"claimed_rate", "true_rate", "overclaims" (claimed but not ok), "missed" (ok but not claimed)}; an empty list gives zeros.
- calibration_gap(runs): claimed_rate − true_rate.
- judge_accuracy(judge, truth): the fraction of positions where an LLM judge's verdict equals the ground truth (0 for empty).
- needs_review(judge, truth): the indices where the judge and the ground truth disagree.
- grade(runs): "trustworthy" if overclaims is 0, otherwise "unreliable"; a self-report is only trusted when it matches the checked state.
Stuck, or something not working? You can come back to it any time.
CHECK YOUR UNDERSTANDING
Is an agent's self-reported success sufficient for grading a task?
Chapter reference: math, worked example & code
This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.
Math and notation
Represent a run as states s0,s1,... and actions a0,a1,... with observed transitions. A budget constraint can bound the sum of measured call costs. History-dependent token growth means a constant per-step estimate is only a simplifying model.
Worked example
A simulated agent may take at most three actions. It moves from room A to B, inspects a clue, then moves to C. The trace contains three actions. If the objective still is not satisfied, the correct result is budget exhausted with partial progress, not a fourth unapproved action or a fabricated success.
Reference code
The following example uses the Python standard library and does not call a remote service.
actions=['move:A->B','inspect:clue','move:B->C']
budget=3
assert len(actions) == budget
objective_satisfied=False
status='complete' if objective_satisfied else 'budget-exhausted'
assert status == 'budget-exhausted'
assert len(actions)*.002 == .006
Python examples are provided for study and local execution; this page does not execute Python.
Common mistakes
A plan is not execution evidence. Reflection can repeat the original mistake. A step budget is useful only if every executed action is counted consistently.