The Parsing Layer: Turning Model Output Into Actions

Most guides to agent output parsing are wrong about the goal. They treat it as an extraction problem — write a smarter regex, bolt on a JSON-repair library, coax structure out of whatever the model emits. That instinct is backwards. The parsing layer's real job isn't to salvage more; it's to reject malformed output loudly and force the model to try again. The best parser is strict, not forgiving. Here's a case study that made that lesson expensive before it made it obvious.

The Problem a Fintech Team Faced

A four-person team at a payments startup built an agent to categorize merchant transactions — assign each charge a category, a confidence, and a flag for review. The model would call a

categorize

Except the agent kept doing quietly wrong things. A transaction would land in the wrong bucket. A confidence score would come back as the string

"high"

The lead engineer described the failure mode perfectly: "It never errors. It just gets the answer subtly wrong, and we find out from an angry customer." That's the trap of forgiving agent output parsing — it converts loud, catchable failures into silent, uncatchable ones.

The Wrong Approach

Their first parser tried to be helpful. It looked something like this:

[object Object], re, json

,[object Object], ,[object Object],(,[object Object],):
    ,[object Object],
    ,[object Object],:
        ,[object Object], json.loads(raw)
    ,[object Object], json.JSONDecodeError:
        cat = re.search(,[object Object],, raw, re.I)
        conf = re.search(,[object Object],, raw)
        ,[object Object], {,[object Object],: cat.group(,[object Object],) ,[object Object], cat ,[object Object], ,[object Object],,
                ,[object Object],: ,[object Object],(conf.group(,[object Object],)) ,[object Object], conf ,[object Object], ,[object Object],}

What this does: it tries to parse clean JSON, and when that fails, it scrapes a category and a number out of whatever text came back — defaulting to

"unknown"

0.5

Every default here is a landmine. A

0.5

0.5

"unknown"

⚠️ Common mistake: Building fallback defaults into your parser. A default value manufactured because parsing failed is a lie your system tells itself. If you can't parse the output, you don't have an answer — you have an error, and it needs to be handled as one, not smoothed over with a reasonable-looking guess.

The Correct Approach: Structured Agent Output Parsing

The fix reframed parsing as validation. The parser's contract became: return a valid, fully-typed object, or raise — never guess. Here's the version that shipped:

[object Object], pydantic ,[object Object], BaseModel, ValidationError, field_validator

,[object Object], ,[object Object],(,[object Object],):
    category: ,[object Object],
    confidence: ,[object Object],
    needs_review: ,[object Object],

,[object Object],
,[object Object],
    ,[object Object], ,[object Object],(,[object Object],):
        ,[object Object], ,[object Object], ,[object Object], <= v <= ,[object Object],:
            ,[object Object], ValueError(,[object Object],)
        ,[object Object], v

,[object Object], ,[object Object],(,[object Object],):
    ,[object Object], Categorization.model_validate_json(raw)  ,[object Object],

What this does: it defines the exact shape and rules the output must satisfy, then validates against them. A string where a float belongs, a confidence of

1.4

The other half of the fix lived in the harness. A raised parse error isn't a crash — it's feedback:

[object Object], ,[object Object],(,[object Object],):
    reply = model.complete(messages, tools=tools)
    ,[object Object], call ,[object Object], reply.tool_calls:
        ,[object Object],:
            parsed = parse_output(call.raw_arguments)
            result = apply(parsed)
        ,[object Object], ValidationError ,[object Object], e:
            result = ,[object Object],
        messages.append({,[object Object],: ,[object Object],, ,[object Object],: call.,[object Object],,
                         ,[object Object],: result})
    ,[object Object], messages

What this does: when parsing fails, it feeds the specific validation error back to the model and lets it correct itself on the next step. The model is remarkably good at fixing

confidence must be between 0 and 1

⚡ Pro tip: Put the schema in the tool description the model sees, and enforce it in the parser. The description reduces how often the model gets it wrong; the parser catches the times it still does. Description without enforcement is hope; enforcement without description is unnecessary retries. You want both.

Results and What Changed

The numbers moved in the direction that matters. Silent miscategorizations — the ones found by customers — dropped to effectively zero, because a malformed output now became a visible retry instead of a plausible-looking wrong answer. The team traded a small increase in average steps per transaction (the occasional re-emit) for the elimination of an entire class of undetectable errors. That's a trade any team handling money takes instantly.

The subtler win was debuggability. With the old forgiving parser, a wrong category left no trace — the parser had already normalized the evidence away. With strict agent output parsing, every failure logged a precise reason: which field, which rule, what the model actually sent. Their mean time to diagnose a parsing issue went from "read two days of transactions" to "read one log line."

⚡ Pro tip: Log every parse failure with the raw model output attached. Over a week, those logs become a map of exactly how your model tends to violate your schema — always the same field, always the same way. That pattern tells you what one sentence to add to your tool description to prevent 90% of the retries.

Bounding the Retry Loop

Strict parsing plus automatic retry has one failure mode you have to design against: the model that can't satisfy the schema, retrying forever. If the task genuinely doesn't fit the shape you've defined, an unbounded retry burns tokens and time chasing an answer that was never possible.

The fintech team capped it explicitly:

[object Object], ,[object Object],(,[object Object],):
    last_error = ,[object Object],
    ,[object Object], attempt ,[object Object], ,[object Object],(max_tries):
        ,[object Object],:
            ,[object Object], parse_output(raw_getter(feedback=last_error))
        ,[object Object], ValidationError ,[object Object], e:
            last_error = e.errors()[,[object Object],][,[object Object],]
    ,[object Object], {,[object Object],: ,[object Object],, ,[object Object],: last_error,
            ,[object Object],: ,[object Object],}

What this does: it retries parsing up to three times, feeding the previous error back into each attempt so the model can correct itself — and if it still fails, it routes the transaction to a human instead of looping forever or inventing an answer. Three attempts caught nearly every recoverable case; the rare true failure became a clean handoff, not a hang.

That last branch matters more than it looks. The whole point of strict agent output parsing is honesty about failure. A bounded retry that ends in "a person should look at this" preserves that honesty while still self-healing the common cases. Unbounded retry quietly trades one silent failure for another — a run that never finishes.

⚡ Pro tip: Track your retry-success curve. If attempt two rescues most failures but attempt three almost never does, drop

max_tries

How to Apply This to Your Situation

You don't need to be handling money for this to matter. The pattern generalizes to any agent whose output feeds a downstream system:

• A legal-tech engineer parsing an agent's contract-clause extractions makes the parser reject any clause missing its source span, so no unsourced claim ever reaches a lawyer's review queue.
• A marketing ops specialist whose agent generates ad variants validates that every variant has a headline under the platform's character limit — a violation raises and retries instead of shipping an ad that gets rejected on upload.
• A biomedical researcher parsing extracted dosages forces units into an enum, so

  "5"

  Copy
  with no unit becomes a loud error rather than an ambiguous number in a dataset humans will trust.

In every case the move is the same: define the strict shape, validate without mercy, and feed failures back as retryable errors. Never let the parser invent.

One more thing the fintech team learned the hard way: schemas drift, and a too-rigid schema becomes its own failure. When they added a new transaction category, the old strict validator rejected every output using it, and the retry loop dutifully burned tokens trying to force the model back to the old set. The fix was to make the enum a warning boundary, not a hard wall — unknown categories were flagged for review rather than rejected outright. Strict parsing means rejecting malformed output, not rejecting new but valid output. The distinction is subtle and it's where a lot of teams over-correct: they make the parser so strict it can't accommodate legitimate change, and legitimate change is the one kind of surprise you actually want to let through.

Next Steps

Start by auditing your current parser for a single word:

default

Strict agent output parsing feels harsher, and it is — that's the point. Loud failures you can fix beat silent failures you ship. The forgiving parser optimizes for "the run completed." The strict parser optimizes for "the run was correct," and only one of those is worth deploying.

The schemas and the schema-embedded tool descriptions that make this work are assets worth keeping. Every agent you build re-uses the same shape of prompt: "here is the exact JSON I expect, here are the rules, emit only that." Saving those proven descriptions in a prompt library like PromptABCD — paired with the Pydantic model they enforce — means your next strict parser starts from a template that already tells the model how to succeed. Build the discipline once, reuse it on every agent after.