Building a Harness That Swaps Models Easily

Teams that hardcode a single provider's API into their agent spend an average of a week rewriting it the first time they want to switch models — and they almost always want to switch, because prices change, a better model ships, or one provider has an outage during a launch. A model agnostic agent harness turns that week into an afternoon by putting a translation layer between your agent logic and any specific provider. The surprising part isn't that this is possible; it's how few teams build it until the rewrite forces them to, when an afternoon of foresight would have saved the whole painful week.

What Is a Model Agnostic Agent Harness?

A model agnostic agent harness is one whose loop, tools, and logic don't depend on any particular model provider. You can point it at one provider today and a different one tomorrow by changing a config value, not by rewriting your agent. The harness talks to an internal interface; an adapter translates that interface to whatever provider you've selected.

The reason this matters more than it first appears is that provider APIs differ in ways that go deeper than syntax. Three major providers express tool calling three different ways — one returns tool calls under one field name, another under a different structure, a third under yet another. Their stop reasons differ. Their message formats differ. These aren't cosmetic differences you can paper over with a find-and-replace; they change how your execution loop reads a response and decides what to do next. Bake one provider's shape into your loop and you've bound your entire agent to that provider.

Here's the adapter interface at the center of it:

[object Object], dataclasses ,[object Object], dataclass

,[object Object],
,[object Object], ,[object Object],:
    text: ,[object Object],
    tool_calls: ,[object Object],      ,[object Object],
    stop_reason: ,[object Object],      ,[object Object],

,[object Object], ,[object Object],:
    ,[object Object], ,[object Object],(,[object Object],) -> ModelReply:
        ,[object Object], NotImplementedError

What this does: it defines one normalized shape every provider gets translated into — a reply with text, a uniform list of tool calls, and a standardized stop reason. Your harness only ever sees

ModelReply

Why It Matters More Than Teams Expect

The obvious benefit is switching providers, but the real value is broader.

Resilience. When a provider has an outage mid-launch — and they do — a model-agnostic harness fails over to a backup provider by changing a string. A hardcoded harness goes down with its provider. That fallback capability has saved more than one product's launch day.

Cost control. Provider pricing shifts, and different models suit different tasks. A model-agnostic harness lets you route cheap tasks to a cheap model and hard tasks to a frontier one, or renegotiate your whole stack when a better deal appears — without touching agent code.

Honest evaluation. With a swappable harness, you can run your own eval suite across five models and pick based on your tasks, not a vendor's benchmark. That's the difference between choosing a model on evidence and choosing it on marketing.

There's a subtler benefit that shows up over a longer horizon: it decouples your product's roadmap from any single provider's. Models improve on their own schedule, and the best model for your task six months from now may come from a provider you're not using today. A hardcoded harness means every such improvement is a rewrite you have to justify and schedule; a model-agnostic one means adopting a better model is a config change you can make the day it ships. Over a couple of years, that difference compounds into whether you're consistently running the best available model or perpetually one migration behind.

⚡ Pro tip: Build the adapter interface before you think you need it, even if you only support one provider at first. Retrofitting model-agnosticism into a harness that assumed one provider's response shape everywhere is the painful week this whole pattern exists to avoid. The interface costs almost nothing to add on day one and a full rewrite to add on day one hundred.

Building the Adapters

Each adapter translates one provider into the normalized shape. The pattern is the same across providers — call the provider, map its response fields into

ModelReply

[object Object], ,[object Object],(,[object Object],):
    ,[object Object], ,[object Object],(,[object Object],): ,[object Object],.client = client
    ,[object Object], ,[object Object],(,[object Object],):
        r = ,[object Object],.client.create(messages=messages, tools=to_provider_a(tools))
        ,[object Object], ModelReply(
            text=r.text,
            tool_calls=[{,[object Object],: c.,[object Object],, ,[object Object],: c.name, ,[object Object],: c.,[object Object],}
                        ,[object Object], c ,[object Object], r.tool_uses],       ,[object Object],
            stop_reason=normalize_stop(r.stop))

What this does: it wraps one provider's client, converts your normalized tool definitions into that provider's expected format on the way in, and maps that provider's response fields — including whatever it calls its tool-call list — back into the uniform

ModelReply

You don't always have to write adapters by hand. Mature routing libraries already translate 100+ providers behind one interface, handling the tool-call-format differences, fallbacks, and retries for you. Using one is often smarter than maintaining your own adapters — the point isn't to write the translation layer yourself, it's to have one so your logic stays provider-independent.

⚡ Pro tip: Don't abstract down to the lowest common denominator. If you only expose features every provider shares, you lose provider-specific advantages like prompt caching or extended context that can dramatically cut cost or improve quality. Instead, expose capabilities as flags —

supports_prompt_caching

max_context

⚡ Pro tip: Keep a tiny "smoke test" that runs one trivial task through every configured adapter and checks that each returns a well-formed

ModelReply

What Model-Agnostic Doesn't Mean

Here's the trap that catches teams who build this well: model-agnostic means your code runs against any provider. It does not mean the same prompt behaves identically across models. Swap the model and the same instructions can produce meaningfully different behavior — different tool choices, different failure modes, different phrasing. The harness is portable; the behavior is not.

This is why a model-agnostic harness makes a per-model eval suite more important, not less. The whole point of easy swapping is to use it, and every swap needs its own evaluation because the new model won't behave like the old one on your tasks. Teams that build the swap capability and skip the per-model evals end up shipping a model change that quietly broke a case the old model handled — the code worked perfectly, and the agent still regressed.

The practical consequence is that "model-agnostic harness" and "model-specific evaluation" are two halves of one discipline. The harness gives you the freedom to swap; the eval suite tells you whether a given swap is actually safe. Build only the first and you have a fast way to ship regressions; build only the second and you have careful evaluation of a model you can't easily change. Together they give you what you actually want — the ability to move to a better model quickly and the confidence that the move didn't break anything. Neither half is optional if swapping is going to be a real capability rather than a latent risk.

⚠️ Common mistake: Assuming that because your harness is model-agnostic, you can swap models freely without re-testing. The code portability lulls you into skipping evaluation, and then a "trivial" provider switch changes behavior on tasks you didn't check. Every model swap is a behavior change that demands its own eval run, even when the code change is a one-line config edit. Portable code, non-portable behavior — hold both ideas at once.

Real Scenarios

The pattern proves itself across teams:

• A startup CTO builds a model-agnostic harness and switches primary providers twice in a year as pricing and quality shift, each time in an afternoon, while competitors who hardcoded are stuck.
• A platform engineer at a SaaS company configures automatic fallback to a second provider, so an outage at the primary provider degrades response quality slightly instead of taking the feature offline.
• A research team runs their agent across four models through one harness, choosing the best per-task based on their own eval numbers rather than published benchmarks.

Conclusion

A model agnostic agent harness is cheap insurance against a future you can't predict — a price change, an outage, a better model — and it costs almost nothing if you build the adapter interface early. Normalize every provider into one internal shape, expose capabilities as flags so you don't lose provider strengths, and remember that portable code doesn't mean portable behavior, so every swap earns a fresh eval.

The prompts and tool descriptions that make your harness work often need small per-model tuning, because the same instruction lands differently across providers. Keeping those model-specific prompt variants organized in a library like PromptABCD — tagged by which model each was tuned for — means switching providers doesn't mean rediscovering how to prompt each one. Your harness stays model-agnostic, and your prompts stay model-aware, which is exactly the combination that makes swapping genuinely painless. Build the interface once, tune the prompts per model, and the next great model that ships is a config change away instead of a quarter away.