Logging and Tracing in an Agent Harness: A Case Study

Picture this: you're an ML engineer on call, and an agent that's been running fine for weeks suddenly starts giving customers wrong answers. You open your logs to find out why — and all you have is the final response the agent gave. Not what tools it called, not what the model saw, not which step went wrong. Just the wrong answer, staring back at you, with no trail leading to it. That's the moment teams discover their agent harness logging was never built for debugging. This is a case study of one team that hit exactly that wall, and what they changed.

The Problem an ML Team Faced

A six-person team ran an agent that answered billing questions by querying internal systems. It worked well enough that they stopped watching it closely. Then a subtle failure crept in: for a specific class of question, the agent started returning outdated numbers. Customers noticed before the team did.

When they went to debug, they had almost nothing. Their logging captured the user's question and the agent's final answer — the two least useful things for diagnosis. They couldn't see which tools the agent called, in what order, with what arguments, or what those tools returned. They couldn't see the actual prompt the model received. The failure was somewhere inside a black box, and their logs illuminated only its edges.

The lead engineer put it bluntly: "We can see that it's wrong. We have no idea where it goes wrong." Reproducing the issue meant re-running the agent and hoping it failed the same way, which it didn't always do. Days went into a bug that better agent harness logging would have surfaced in minutes.

The Wrong Approach

Their original logging looked reasonable on the surface:

[object Object], logging

,[object Object], ,[object Object],(,[object Object],):
    result = agent_loop(task, tools, model)
    logging.info(,[object Object],)
    ,[object Object], result

What this does: it logs the input and the output of the whole run as a single line. For a stateless function this would be fine. For a multi-step agent it's nearly useless, because everything interesting happens between the input and the output — and none of that is captured. The loop could have called five tools, hit two errors, and re-read the same file three times, and this log would show none of it.

The deeper problem is that agents fail in the middle. A wrong final answer is a symptom; the cause is a specific tool call that returned stale data, or a model decision made on a truncated result. Logging only the endpoints is like debugging a crash with only the program's exit code.

The Correct Approach: Structured Agent Harness Logging

The fix was to log every step as structured data, with a shared trace ID tying a run's events together. Not prose log lines — queryable records.

[object Object], json, time, uuid

,[object Object], ,[object Object],(,[object Object],):
    trace_id = ,[object Object],(uuid.uuid4())
    ,[object Object], ,[object Object],(,[object Object],):
        rec = {,[object Object],: trace_id, ,[object Object],: time.time(), ,[object Object],: event, **fields}
        ,[object Object], ,[object Object],(log, ,[object Object],) ,[object Object], f:
            f.write(json.dumps(rec) + ,[object Object],)

    emit(,[object Object],, task=task)
    messages = [{,[object Object],: ,[object Object],, ,[object Object],: task}]
    ,[object Object], step ,[object Object], ,[object Object],(,[object Object],):
        t0 = time.time()
        reply = model.complete(messages, tools=,[object Object],(tools))
        emit(,[object Object],, step=step, latency=time.time() - t0,
             prompt_tokens=reply.usage.,[object Object],, completion_tokens=reply.usage.output,
             tool_calls=[c.name ,[object Object], c ,[object Object], reply.tool_calls])
        ,[object Object], reply.stop_reason == ,[object Object],:
            emit(,[object Object],, step=step, answer=reply.text[:,[object Object],])
            ,[object Object], reply.text
        ,[object Object], call ,[object Object], reply.tool_calls:
            r = tools[call.name](**call.args)
            emit(,[object Object],, step=step, tool=call.name, args=call.args,
                 result_preview=,[object Object],(r)[:,[object Object],])
            messages.append({,[object Object],: ,[object Object],, ,[object Object],: call.,[object Object],, ,[object Object],: ,[object Object],(r)})

What this does: it assigns each run a trace ID, then records a structured event for every model reply and every tool result — including latency, token counts, tool names, arguments, and a result preview. Now a run's entire history is queryable. Finding the stale-data bug became a matter of filtering the trace for the tool call that returned old numbers, which took a single grep instead of days.

⚡ Pro tip: Log the exact prompt sent to the model, including the rendered system prompt and the tool schemas, at least in a debug mode. This is the thing that actually varies and breaks — a one-word change to a tool description, an accidentally doubled instruction — and it's invisible if you only log the user's question. When an agent's behavior shifts for no apparent reason, the rendered prompt is usually where the answer hides.

Results and What Changed

The stale-data bug was found in an afternoon once the traces existed. The trace showed the agent calling a cache-reading tool that a recent change had pointed at an outdated store — obvious the moment the tool result was visible, invisible while it wasn't. The fix was one line; finding it was the whole battle, and structured logging won it.

The lasting change was cultural. With per-step traces, the team could ask questions they couldn't before: which tools are slowest, which steps burn the most tokens, how often the agent retries. Their token cost per run dropped noticeably once they could see that one tool was returning huge results that inflated every subsequent model call. You can't optimize what you can't measure, and the trace made the run measurable for the first time.

⚡ Pro tip: Include token counts and latency on every step event, even when you're not chasing a cost problem yet. When you eventually are — and with agents, you eventually are — you'll have weeks of history to analyze instead of having to add the instrumentation and wait. The cheapest time to add cost logging is before you need it.

Making Traces Portable With Spans

Once your traces prove their worth locally, the next step is a shape that survives beyond one machine's log file. The idea that carries over is the span: a nested record with a start, an end, and a parent. A run is a span; each step is a child span; each tool call is a child of the step. The nesting reconstructs the whole tree of what happened and how long each part took.

[object Object], ,[object Object],(,[object Object],):
    ,[object Object], {,[object Object],: trace_id, ,[object Object],: ,[object Object],,
            ,[object Object],: parent, ,[object Object],: time.time()}

What this does: it creates a span that names its own place in the tree — which run, which step, whose child — so a viewer can rebuild the full hierarchy and show you where the time and tokens went at a glance. This is the same model that standard tracing systems use, which means once your events are span-shaped, you can export them to an existing observability backend instead of building your own viewer. You get flame-graph views of agent runs for the cost of formatting your events consistently.

The payoff is that your agent's traces sit alongside the rest of your system's traces — the database calls, the API requests — under one trace ID. When an agent is slow because a downstream service is slow, one correlated view shows both, instead of two separate investigations that never quite connect.

How to Apply This to Your Situation

The pattern fits any agent whose behavior you need to explain after the fact:

The pattern fits any agent whose behavior you need to explain after the fact:

• A fintech engineer logs a structured trace per run and retains it for audit, so when a regulator asks why an agent made a particular decision, the full tool-by-tool trajectory is on record.
• A customer-support platform lead correlates traces with customer complaints via the trace ID, turning "the bot was wrong" tickets into exact reproductions instead of guesswork.
• A research engineer benchmarking agent variants uses the token and latency fields to compare not just accuracy but efficiency across versions, catching a "more accurate but three times more expensive" regression that final-answer logging would have hidden.

The move is always the same: emit a structured event per step, tie them with a trace ID, and capture what the model actually saw. Prose logs tell stories; structured traces answer questions.

⚡ Pro tip: Redact secrets at the logging boundary, not after. If a tool result or prompt might contain an API key, a token, or personal data, scrub it in the

emit

⚠️ Common mistake: Logging so much that you never look at any of it. The opposite of logging only the final answer is dumping every token of every message on every step into an unqueryable text blob — technically complete, practically useless. The fix isn't less data, it's structure: log rich fields, but as JSON you can filter, not prose you have to read. A trace you can query with a one-line filter beats a trace you have to scroll through, every time. Aim for events you can slice by trace ID, step, tool, or token count in seconds — if answering "which step was slowest" takes more than one query, your logging has structure problems, not volume problems.

There's a retention dimension too. Full traces with rendered prompts get large fast, so keep detailed traces for a short window — long enough to debug a fresh incident — and downsample older runs to just the summary events. You almost never need the full rendered prompt from six weeks ago, but you very often need last night's. Match your retention to how you actually debug.

Next Steps

Audit your own logging by asking one question: if your agent gave a wrong answer right now, could you tell which step caused it? If the answer is no, you have this team's problem. Add per-step structured events with a trace ID today, then add token and latency fields, then log the rendered prompt in debug mode. Each addition turns a future black-box failure into a solvable one.

Good agent harness logging is the difference between "we know it's wrong" and "we know exactly where." The trace schema and the debug-mode prompt-logging setup that make this work are reusable across every agent you build. Keeping those logging patterns — and the annotated prompts you capture from real traces — in a prompt library like PromptABCD means your next agent is observable from its first run, and the hard-won lesson about logging the rendered prompt doesn't have to be relearned one outage at a time.