Building an Evaluation Harness for Your Agent

Most guides to building an evaluation harness reach for the wrong tool first: they tell you to grade agent outputs with another language model. LLM-as-judge, they say, and off you go asking a model whether the answer was good. That instinct produces evals you can't trust, because you've replaced a hard measurement problem with a softer, noisier one. The better move — when you can build agent eval harness infrastructure around it — is to make your tasks verifiable, so a plain function decides pass or fail. Let's tear down the judge-first approach and rebuild around verification.

Before: The LLM-as-Judge Reflex

Here's the eval harness people build first, because it feels general:

[object Object], ,[object Object],(,[object Object],):
    verdict = judge_model.complete(
        ,[object Object],
        ,[object Object],)
    ,[object Object], ,[object Object], ,[object Object], verdict.text

What this does: it asks a model to judge whether the agent succeeded. It works on any task without you writing a checker, which is exactly why it's tempting — one grader for everything. The problem is what you've traded away: the judge is itself nondeterministic, has its own biases, and can be wrong in ways that correlate with the agent's own errors. You're measuring your agent with a ruler made of rubber.

Why That Choice Backfires

An LLM judge fails in specific, compounding ways that undermine the eval's whole purpose.

It's inconsistent. Ask it twice and you may get different verdicts on the identical output. An eval whose grader is noisy can't isolate whether a score change came from the agent or the judge. You've added a second source of variance to a measurement that already had too much.

It's biased toward plausible-sounding answers. A confident, well-formatted wrong answer often gets a PASS from a judge that's pattern-matching on tone, while a terse correct answer gets dinged. Your eval then rewards the wrong thing — style over substance — and your agent drifts toward sounding right rather than being right.

And it's expensive and slow. Every eval run now costs a second model call per task, which discourages running evals often — and an eval you run rarely is an eval that stops catching regressions. The judge quietly makes the eval too costly to use the way it needs to be used.

⚠️ Common mistake: Reaching for LLM-as-judge as the default grader instead of the last resort. Judges have a place — for genuinely subjective qualities like tone or helpfulness — but using one for anything a function could check trades a reliable measurement for an unreliable one. If success can be defined precisely, define it precisely. Save the judge for the cases that truly resist programmatic checking, and even then, validate the judge against human labels first.

After: Verifiable Tasks and Programmatic Graders

The rebuild starts upstream, in how you design tasks. Make each task's success machine-checkable:

TASKS = [
    {,[object Object],: ,[object Object],,
     ,[object Object],: ,[object Object],,
     ,[object Object],: ,[object Object], env: env.read(,[object Object],).get(,[object Object],) == ,[object Object],},
    {,[object Object],: ,[object Object],,
     ,[object Object],: ,[object Object],,
     ,[object Object],: ,[object Object], env: env.run(,[object Object],).returncode == ,[object Object],},
    {,[object Object],: ,[object Object],,
     ,[object Object],: ,[object Object],,
     ,[object Object],: ,[object Object], output: output == ,[object Object],},
]

,[object Object], ,[object Object],(,[object Object],):
    report = {}
    ,[object Object], t ,[object Object], tasks:
        passes = ,[object Object],(,[object Object],(t[,[object Object],](agent(t[,[object Object],]))) ,[object Object], _ ,[object Object], ,[object Object],(runs))
        report[t[,[object Object],]] = passes / runs
    ,[object Object], report

What this does: each task carries a

check

The shift in effort is the whole point. With a judge, the runner is easy and the grading is unreliable. With verifiable tasks, the grading is trivial and the task design is where you invest — crafting tasks whose success is unambiguous. That's real work, but it's work that produces a trustworthy number, and a trustworthy number is the only kind worth having. The teams that make this trade never go back — once you've had an eval you fully believe, a fuzzy one feels like flying blind.

⚡ Pro tip: Model your verifiable tasks on how published coding benchmarks grade — apply the agent's change, then run a test suite that must pass. "Did a specific test go from failing to passing" is the gold standard of a verifiable check, because it can't be faked by a plausible-sounding answer. If your domain has anything test-shaped, lean on it.

Breaking Down a Verifiable Eval Harness

Three components make the rebuilt harness work, and separating them keeps it maintainable.

The task set — inputs paired with machine-checkable success criteria. This is your most valuable asset and the hardest to build well. Each task should have exactly one clear notion of success.

The runner — executes the agent against each task in a clean, isolated environment so one task can't contaminate the next. Reset state between runs, or a task that mutates the filesystem quietly changes the starting conditions for everything after it.

The grader — applies each task's check and aggregates. With verifiable tasks this is a few lines, because the intelligence lives in the check functions, not in a scoring model.

For the small set of genuinely subjective tasks, a judge is acceptable — but validate it first against human labels, and report those tasks separately so their softer grading doesn't blur your hard numbers.

The order of these components in your build matters too. Start with the grader — the check functions — because writing the check forces you to define success precisely, and a task whose success you can't write as a function is a task you don't actually understand yet. Teams that start with the runner tend to end up with lots of tasks and vague grading; teams that start with the checks end up with fewer tasks but a number they can trust. Define "correct" as code first, and the rest of the harness falls into place around it.

⚡ Pro tip: Give every task a clean, disposable environment — a fresh temp directory, a reset database fixture. The most confusing eval bug is a task that passes alone but fails in the suite, and the cause is almost always state leaking from an earlier task. Isolation makes your scores reproducible, which is the entire reason the eval exists.

How Many Tasks Does a Trustworthy Eval Need?

A question that stalls people when they build agent eval harness suites: how many tasks are enough? The honest answer is that it depends on how confident you need to be, and a little arithmetic beats a gut guess. With ten tasks, the difference between a 70% and an 80% agent is inside the noise — you can't distinguish them reliably. With a hundred tasks run several times each, that same gap becomes a clear, defensible signal.

Start smaller than a hundred, though, or you'll never start. A tight set of twenty well-chosen, verifiable tasks that each probe a distinct capability tells you more than two hundred near-duplicates that all test the same easy path. Coverage of different behaviors matters more than raw count early on. Add tasks as real failures teach you what you weren't testing — every production bug that surprised you becomes a new task, and the suite grows toward your actual risk surface instead of toward an arbitrary number.

[object Object], ,[object Object],(,[object Object],):
    ,[object Object], collections ,[object Object], Counter
    by_cap = Counter(t.get(,[object Object],, ,[object Object],) ,[object Object], t ,[object Object], tasks)
    ,[object Object], ,[object Object],(by_cap)   ,[object Object],

What this does: it counts how many tasks probe each capability, surfacing the gaps — an "untagged" pile or a capability with only one task is a blind spot. Coverage by capability is a better health check for an eval suite than the total task count, because it shows you what you aren't measuring.

⚡ Pro tip: When you can't decide whether to add a task, ask "would this catch a bug the existing tasks miss?" If yes, add it. If it just re-tests a path you already cover, skip it — a redundant task inflates your suite's runtime without improving its signal. Optimize for distinct failure coverage, not for a big number.

Variations for Different Contexts

The verification strategy bends to the domain:

• A backend engineer builds tasks where the check runs the project's real test suite after the agent's patch, mirroring how production correctness is actually defined.
• A data engineer writes checks that validate the schema and row counts of an agent's output table, so "did it transform the data correctly" becomes a precise assertion rather than a judgment call.
• A conversational-AI designer uses programmatic checks for factual accuracy and reserves a validated judge only for tone, reporting the two separately so a soft tone score never inflates a hard correctness number.

Same harness, different check functions — the architecture holds while the verification adapts to what "correct" means in each field.

Save and Reuse This

When you build agent eval harness infrastructure, resist the judge-first reflex. Design verifiable tasks, grade them with functions, and reserve LLM judgment for the narrow band of qualities that genuinely resist a precise check. The result is an eval you can run constantly and trust completely, instead of one that's cheap to write and impossible to believe.

The task definitions and check functions you craft are the heart of the whole system, and they're worth keeping organized and reusable across agent versions. Storing the task prompts and their success criteria in a library like PromptABCD — tagged by the capability each verifies — means your eval suite survives rewrites and model swaps intact. You invest once in tasks whose success is unambiguous, and every future change gets measured against a bar that doesn't wobble. That stable bar is what lets you move fast without breaking things quietly — the whole reason to build the harness in the first place.