Managing the System Prompts Your Harness Injects

A one-word change to a tool description once broke every task in an agent I maintained. Someone edited a system prompt string inline — swapping "list" for "array" in a tool's description — to fix one failing case, and the change silently altered how the model routed across dozens of unrelated tasks. Nobody caught it for a day, because the system prompts lived as scattered string literals with no versioning, no review, and no tests. This is what harness system prompt management exists to prevent, and most teams don't build it until a change like that costs them a day. Let's tear down the scattered-strings approach and rebuild around versioned, tested prompts.

Before: System Prompts as Scattered Strings

Here's how system prompts usually live in a young harness:

[object Object], ,[object Object],():
    system = ,[object Object],  ,[object Object],
    tools = [
        {,[object Object],: ,[object Object],, ,[object Object],: ,[object Object],},
        {,[object Object],: ,[object Object],, ,[object Object],: ,[object Object],},
    ]  ,[object Object],
    ,[object Object], Agent(system=system, tools=tools)

,[object Object],

What this does: it embeds the system prompt and every tool description as string literals right in the code, often duplicated across files with small unintended variations. It works, and it's how nearly everyone starts. It's also a landmine, because these strings are the actual interface between you and the model's behavior — and you're treating them like throwaway text instead of the load-bearing configuration they are.

Why That Approach Fails

Scattered inline prompts fail in ways that compound as the agent grows.

They drift. The same system prompt gets copied into three places, then edited in one, and now your agent behaves differently depending on which entry point built it. Nobody decided that; it happened through ordinary editing. You end up with several slightly different agents wearing the same name.

They change without review. A prompt string is just code, so anyone can edit it in a hurry to fix one case — and a prompt change can ripple across every task, because the system prompt and tool descriptions shape all the model's behavior, not just the case being fixed. The one-word edit that broke my agent's routing was a "trivial" change nobody thought needed review.

And they can't be tested in isolation. When behavior regresses, you can't tell whether the code changed or a prompt changed, because they're the same commit with no separation. The prompt — the thing most likely to have shifted behavior — is invisible as its own variable.

Good harness system prompt management fixes each of these directly. It gives every prompt one home so it can't drift, puts changes behind review so nobody edits the model's behavior in a hurry, and separates prompts from code so a regression can be traced to the exact prompt version that caused it. The scattered-strings approach fails precisely because it treats the most behavior-critical text in your whole system as if it were an afterthought.

⚠️ Common mistake: Treating tool descriptions as documentation rather than as behavior. A tool's description isn't a comment for humans — it's the instruction the model routes on. Change "Search the database for records" to "Search records" and you may change which tool the model picks and when. Because the description drives routing, every edit to it is a behavior change that deserves the same care as a code change, not a casual wording tweak made in passing.

After: Versioned, Centralized Prompt Management

The rebuild pulls prompts out of the code into versioned, named artifacts with variables, so every prompt has one source of truth and a history.

PROMPTS = {
    ,[object Object],: {
        ,[object Object],: ,[object Object],
                    ,[object Object],,
        ,[object Object],: ,[object Object],,
        ,[object Object],: ,[object Object],,
    },
    ,[object Object],: {
        ,[object Object],: ,[object Object],
                       ,[object Object],,
        ,[object Object],: ,[object Object],,
    },
}

,[object Object], ,[object Object],(,[object Object],):
    ,[object Object], PROMPTS[prompt_id][,[object Object],].,[object Object],(**,[object Object],)

What this does: it stores each prompt and tool description as a named, versioned entry with a template and metadata about which eval it was tested against. There's one source of truth per prompt, changes bump a version, and the tie to a golden eval means you know a prompt version was validated. Rendering fills in variables at runtime, so there's no duplication to drift.

The second half is treating a prompt change like a reviewed code change with its own eval:

[object Object], ,[object Object],(,[object Object],):
    ,[object Object], eval_result[,[object Object],] < ,[object Object],:
        ,[object Object], ValueError(,[object Object],
                         ,[object Object],)
    log_promotion(prompt_id, new_version, eval_result, approver)

What this does: it refuses to promote a new prompt version unless it passed its eval, and it records who approved it. A prompt change now runs the eval suite before it ships, so the one-word edit that would have quietly broken routing instead fails the eval and never reaches production. The prompt is gated exactly like code, because it is the thing that most directly controls behavior.

Breaking Down Prompt Management

Three practices make this work, and each closes one of the failure modes above.

One source of truth per prompt. Every prompt lives in exactly one place, referenced by ID everywhere it's used. This kills drift — there's nothing to fall out of sync because there's only one copy.

Version the tool schemas alongside the system prompt. The tool descriptions are part of the behavioral contract, so version them with the system prompt, not separately. A change to either is a change to how the agent behaves, and they should move together through review.

Test every prompt version against a golden eval before promoting. A prompt change is a behavior change, so it earns the same eval a code change would. Tie each prompt version to the eval it passed, and never promote one that regressed.

These three practices reinforce each other. One source of truth makes versioning meaningful, versioning makes eval-gating possible, and eval-gating makes review on diffs actionable — a reviewer can look at a prompt change and the eval result together. Adopt them piecemeal and each feels like bureaucracy; adopt them together and they form a pipeline where a prompt change flows from edit to eval to review to promotion, with a gate at each step. That pipeline is what makes editing a prompt feel as safe as editing code, which is the entire goal — because right now, for most teams, it's considerably more dangerous.

⚡ Pro tip: Diff prompt versions like code and require review on the diff. When someone changes a tool description, a reviewer should see the before and after and ask "how does this change routing?" That single question, asked on every prompt diff, catches the one-word changes that quietly ripple across every task — the exact failure that's nearly impossible to catch any other way.

Variations for Different Contexts

The right rigor scales with stakes:

• A solo developer can keep prompts in a single versioned file and eyeball changes, which already beats scattered inline strings by a wide margin.
• A platform team running many agents needs full management — IDs, versions, eval gates, review on diffs — because a prompt shared across agents means one edit affects them all.
• A regulated-industry team versions every prompt with an approver and an eval record, so they can show auditors exactly which prompt version was live when a given decision was made.

The through-line is that the rigor tracks how many things depend on the prompt. A solo developer's prompt affects one agent they fully understand, so lightweight versioning suffices. A platform team's prompt might be shared across a dozen agents, so a change nobody reviewed can ripple into behavior far from where it was edited — which is exactly the scenario that justifies the full pipeline. Match the ceremony to the blast radius: the more agents and people a prompt touches, the more the versioning, eval-gating, and review earn their keep.

⚡ Pro tip: Store the rendered prompt with each run's logs, not just the prompt ID. When you debug a past run, you want to see the exact text the model received, variables filled in — because the bug might be in how a variable rendered, not in the template itself. The ID tells you which template; the rendered text tells you what actually happened.

⚡ Pro tip: When you retire a prompt version, keep it, don't delete it. An old version is the fastest way to answer "did this behavior change when we updated the prompt?" — you can render both versions against the same task and compare. A prompt history you can diff and replay is worth far more than a tidy file with only the current version, and the storage cost of keeping every version is trivial next to the debugging time it saves.

Save and Reuse This

Harness system prompt management turns your prompts from scattered, unreviewed string literals into versioned, tested artifacts that change deliberately instead of by accident. One source of truth per prompt, tool schemas versioned alongside the system prompt, an eval gate before promotion, and review on every diff — together they prevent the one-word change that silently breaks every task. Prompts are the interface to the model's behavior; managing them like the critical configuration they are is not overhead, it's the whole game.

This is exactly what a prompt library is built for. Keeping your system prompts and tool descriptions in PromptABCD — versioned, tagged by which harness and model they belong to, with the eval each version passed recorded alongside — gives you the single source of truth, the version history, and the review surface this whole approach depends on. Manage your prompts as deliberately as your code, and the day-long mystery of "which trivial edit broke everything" simply stops happening — because there are no trivial edits to the thing that controls your agent's behavior, only reviewed, tested, versioned ones.