A1.5 · LESSON 5 OF 6

Structured outputs

Trace a model-requested tool call through schema validation, execution and a returned observation.

THE IDEA

Structured outputs

A schema can make the model's output well-shaped: parseable JSON with the right keys and types. It cannot make it true. Reading "Two blue mugs at 8.50 each" as quantity 3 passes every shape check. So validate in layers: parse, check shape, then check domain constraints and compare values with the source. Treat anything coming from tools or documents as data, never as new authority over your application.

source: "Two blue mugs at 8.50 each, ship to Lyon." model output: {"item":"blue mug","quantity":3,"unit_price":8.5}

quantity: unit price:

1. parses as JSON: yes 2. shape (types and ranges): pass 3. matches the source (quantity 2, price 8.5): quantity_mismatch implied total 25.5 correct total 17

Passing a schema proves only the shape. Quantity 3 for "two mugs" is valid JSON and still wrong, so check values against the source and treat tool output as data, not as new instructions.

Change the quantity and price and watch which check catches it.

Predict first. Can valid JSON contain an unsupported claim?

Chapter context

A tool call is a structured request for the application to perform an operation. The model proposes arguments; the surrounding application decides whether the operation is valid and permitted and is responsible for executing it.

TRY IT YOURSELF

Live lab

Extract an order from a sentence as JSON. This check does three things: does it parse, are the values in range, and do they match what the source text actually says?

TRY IT YOURSELF

Valid is not the same as true

Extract an order from a sentence as JSON. This check does three things: does it parse, are the values in range, and do they match what the source text actually says?

Your goal

Get JSON with item, quantity and unit_price where the values match the sentence, not just the format.

Your model

ProviderModel

API keyRemember on this device

Your key is stored only in this browser and goes straight to Anthropic (Claude). Coursefy never receives it. Use a restricted key with a low spending limit (create one ↗). Without a key you can still view one sample response.

System prompt (optional)User message

Extract the order from this text: "Two blue mugs at 8.50 each, ship to Lyon." Return the item, quantity and unit_price.

max output tokens 120

This request as code

import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

message = client.messages.create(
    model="claude-haiku-4-5",               # Model
    max_tokens=120,                 # max output tokens
    messages=[{"role": "user", "content": "Extract the order from this text: \"Two blue mugs at 8.50 each, ship to Lyon.\" Return the item, quantity and unit_price."}],  # User message
)

print(message.content[0].text)
print(message.stop_reason)   # end_turn or max_tokens
print(message.usage.input_tokens, message.usage.output_tokens)

Update the controls above and this code changes with them. Run it yourself with the key in an environment variable.

No key yet, so this shows one sample response for the starting prompt. Add a key above to experiment for real.

Exercises

Part 1. Do it hands-on

Compute the correct and the claimed total, then write the layered validator.

source: "Two blue mugs at 8.50 each, ship to Lyon." model output: {"item":"blue mug","quantity":3,"unit_price":8.5}

quantity: unit price:

1. parses as JSON: yes 2. shape (types and ranges): pass 3. matches the source (quantity 2, price 8.5): quantity_mismatch implied total 25.5 correct total 17

Passing a schema proves only the shape. Quantity 3 for "two mugs" is valid JSON and still wrong, so check values against the source and treat tool output as data, not as new instructions.

correct total total implied by quantity 3 passes the shape check? (yes/no) valid JSON can be wrong? (yes/no)

To do: Source text: "Two blue mugs at 8.50 each, ship to Lyon." The correct order total (quantity × unit price, 2 decimals)?
To do: A model returns {"item": "blue mug", "quantity": 3, "unit_price": 8.5}. The total implied by that JSON?
To do: Does that JSON pass a shape check (keys present, quantity a positive integer, price a number)? Type yes or no
To do: Can valid JSON contain an unsupported claim? Type yes or no

Part 2. Do it in code

parse_json_safe(text): the parsed object, or None if the text is not valid JSON (strictly: no surrounding prose).
shape_ok(order): True if order is a dict with item (a non-empty string), quantity (an int, not a bool, at least 1) and unit_price (an int or float, not a bool, at least 0).
source_facts(text): {"quantity": ..., "unit_price": ...} read from a sentence like "Two blue mugs at 8.50 each, ship to Lyon.": the first word as a number (one … ten as words, or digits) and the number before " each" as a float; None for a fact that is not found.
problems(order, text): ["shape"] if shape_ok is False. Otherwise "quantity_mismatch" if the quantity differs from the source and then "price_mismatch" if the unit price differs by more than 1e-9.
total(order): quantity × unit_price rounded to 2 decimals.

import json
import re

WORDS = {"one": 1, "two": 2, "three": 3, "four": 4, "five": 5, "six": 6, "seven": 7, "eight": 8, "nine": 9, "ten": 10}

def parse_json_safe(text):
    pass

def shape_ok(order):
    pass

def source_facts(text):
    pass

def problems(order, text):
    pass

def total(order):
    pass

Ctrl/⌘ + Enter runs it

Both the live lab and the exercises must be solved to finish this lesson.

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Can valid JSON contain an unsupported claim?

AYesBNo

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

A useful execution model is: requested operation + validated arguments + permission decision → execution result. Schema validation establishes a data-shape contract; authorization establishes whether this caller may perform this operation. Neither implies the other.

Worked example

A calculator tool requires numeric a and b and returns a+b. For a=2,b=3, its result is 5. If the request instead asks for an unknown field or omits b, reject or repair the request according to the contract before executing. A retry of this pure calculation is harmless; that does not establish the same property for a side-effecting tool.

Reference code

The following example uses the Python standard library and does not call a remote service.

def add(a,b):
    if type(a) not in (int,float) or type(b) not in (int,float):
        raise ValueError('numeric inputs required')
    return a+b
assert add(2,3) == 5
try:
    add('2',3)
except ValueError:
    pass
else:
    raise AssertionError('invalid input accepted')

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

A model proposal is not permission. JSON schema shape validation is not business-rule validation. Tool outputs must not silently become trusted instructions.