Tool Routing Inside an AI Harness: A Practical Guide

An agent I was debugging last year kept deleting the wrong files. The task was "clean up stale build artifacts." Instead, it ran delete on source files, over and over, cheerfully reporting success. The model wasn't broken. The prompt was fine. The failure was in agent tool routing — the harness had two tools, delete_artifact and delete_file, with nearly identical descriptions, and the model kept picking the general one. One overlapping sentence in a tool description cost an afternoon and a git reflog rescue.

That's the thing about routing: when it fails, it fails silently and confidently. Let's make it fail loudly instead.

What Is Agent Tool Routing?

Agent tool routing is the step where the harness takes the model's requested action and maps it to an actual function to run. The model says "I want to call search with these arguments"; the router decides which search that is, whether it exists, whether it's allowed right now, and what to do if any of those checks fail.

In the simplest harness, routing is a dictionary lookup:

[object Object], ,[object Object],(,[object Object],):
    fn = tools.get(call.name)
    ,[object Object], fn ,[object Object], ,[object Object],:
        ,[object Object], ,[object Object],
    ,[object Object], fn(**call.args)

What this does: it finds the requested tool by name and runs it, or returns an error if the name doesn't exist. This is correct and complete for a three-tool agent. It's also where every routing bug you'll ever hit starts, because the dictionary lookup is the easy 20% — the other 80% is everything the lookup doesn't check.

Why Routing Deserves More Attention Than It Gets

Most people assume routing failures mean "the model picked the wrong tool." In practice, that's the rare case. The common failures are subtler:

• Right tool, wrong arguments. The model picks send_email correctly but passes to="the customer" instead of an address. The lookup succeeds; the call fails downstream.
• Overlapping descriptions. Two tools whose descriptions share phrasing — like my delete_artifact and delete_file — and the model can't reliably tell them apart. The router runs exactly what it was told; the ambiguity was upstream.
• Tool available at the wrong time. The model calls deploy before run_tests has passed. The tool exists, so the router runs it, and now broken code is in production.

Notice that a plain dictionary lookup catches none of these. It only checks existence. The tool it ran was real, so it reported success — while doing the wrong thing. That gap between "the tool exists" and "the tool should run, with these arguments, right now" is where agent tool routing earns real engineering attention.

⚡ Pro tip: When an agent misbehaves, check your tool descriptions before you touch the routing code. Ambiguous behavior almost always traces to two descriptions that overlap or one description that under-specifies. The router is usually doing its job correctly; the job was just described badly.

How Does Agent Tool Routing Work Beyond a Dictionary?

A production router does three jobs the lookup skips: it validates arguments, it detects ambiguity before it bites, and it gates tools by state. Here's a router that does all three:

[object Object], ,[object Object],(,[object Object],):
    spec = tools.get(call.name)
    ,[object Object], spec ,[object Object], ,[object Object],:
        ,[object Object], ,[object Object],

    ,[object Object],
    ,[object Object], call.name ,[object Object], spec.requires ,[object Object], ,[object Object], state.get(spec.requires[call.name]):
        ,[object Object], (,[object Object],
                ,[object Object],)

    ,[object Object],
    ,[object Object],:
        args = schemas[call.name].model_validate(call.args)
    ,[object Object], Exception ,[object Object], e:
        ,[object Object], ,[object Object],

    ,[object Object],
    ,[object Object], spec.fn(**args.model_dump())

What this does: before running anything, it checks whether the tool is permitted in the current state (so deploy can't fire until tests_passed is true), validates the arguments against a schema, and only then executes. Each guard returns a specific, model-readable error instead of running the wrong thing successfully.

State-gating is the part almost no tutorial covers, and it's the one that prevents the scariest failures. By attaching a requires map to each tool — deploy requires tests_passed, refund requires manager_approved — you encode your workflow's real ordering into the router itself. The model can ask for deploy early; the router simply won't let it, and tells it why. That's a guardrail the prompt alone can't reliably enforce, because prompts are suggestions and routers are law.

⚡ Pro tip: Measure description overlap before you ship. A quick embedding-similarity check across your tool descriptions surfaces the pairs the model will confuse. If two descriptions score above ~0.85 similarity, rewrite one to be sharply distinct — lead with the difference, not the shared purpose. "Deletes build outputs in ./dist only" beats "Deletes files you don't need."

Naming and Namespacing Tools

The cheapest routing improvement is naming discipline, and it's underrated. Tools that share a prefix and a clear scope are easier for both the model and you to route correctly:

What this does: the namespace signals category at a glance, so the model groups related actions and you can apply category-wide rules — like "everything under db. requires a readonly=False flag to write." Namespacing also makes state-gating readable: deploy.release requires ci.passed documents your pipeline in the tool table itself.

Three teams where careful routing changed outcomes:

• A DevOps engineer state-gates rollback behind a captured healthy snapshot, so the agent can't roll back to a state that was never known-good.
• A healthcare software developer namespaces phi.read separately from public.read and routes every phi. call through an extra audit-logging wrapper, satisfying a compliance requirement in the router rather than in every tool.
• A game studio's build engineer adds argument validation to upload_build so the model can't pass a platform string the store doesn't recognize — the router rejects it with the list of valid platforms, and the model self-corrects.

Routing When the Model Asks for Nothing

There's a routing case that isn't about picking between tools — it's the model returning plain text when you expected a tool call, or calling no tool at all mid-task. A naive harness treats "no tool call" as "done" and exits, sometimes leaving the task half-finished.

A sturdier router distinguishes finished from stuck:

[object Object], ,[object Object],(,[object Object],):
    ,[object Object], reply.tool_calls:
        ,[object Object], ,[object Object],
    ,[object Object], state[,[object Object],]:
        ,[object Object], ,[object Object],
    ,[object Object],
    ,[object Object], ,[object Object],  ,[object Object],

What this does: it treats an empty tool-call list as three different situations depending on state — act on requested tools, exit only if completion was explicitly signalled, and otherwise nudge the model back to work instead of silently ending a half-done task. That nudge branch catches the quiet drop-outs where a model trails off mid-plan.

The nudge should be specific, not scolding. "You stopped after reading the file but haven't written the summary yet — call write_summary or finish()" works far better than a generic "continue." You're routing the model back to the missing step, and naming the step is what makes the re-prompt land.

⚡ Pro tip: Count how often your agent hits the nudge branch. A high nudge rate usually means your finish() tool is missing or badly described — the model wants to stop but has no clean way to say so, so it just goes quiet. Give it an explicit, well-described exit and the silent drop-outs mostly vanish.

Common Mistakes

⚠️ Common mistake: Registering two tools whose descriptions overlap and then blaming the model when it picks wrong. The model routes on the descriptions you give it. If delete_file and delete_artifact read almost the same, the model's confusion is a specification bug, not a reasoning failure. Either merge the tools and disambiguate with an argument, or rewrite the descriptions so the boundary between them is unmistakable.

A few more routing traps worth naming:

• No argument validation. Existence checks aren't enough. A tool that runs with garbage arguments fails in ways that are far harder to trace than a clean ARG ERROR at the router.
• Ignoring state. Without state-gating, ordering lives only in the prompt, and prompts don't reliably enforce ordering. Encode "X before Y" in the router where it's actually binding.
• Silent success on the wrong tool. The worst outcome isn't an error — it's a successful call that did something you didn't want. Make your router prefer a loud rejection over a quiet mistake every time.

Conclusion

Agent tool routing is more than a dictionary lookup, even though the lookup is where it starts. The real work is in the guards around it: validating arguments so bad calls fail clean, checking descriptions so the model can tell tools apart, and gating tools by state so your workflow's ordering is enforced by code instead of hoped for in a prompt. Get those three right and a whole category of confident, silent failures disappears.

The habit that ties it all together is making the router observable. Every routing decision — which tool, which arguments, which guard passed or blocked it — should land in a log you can read after the fact. When my agent deleted the wrong files, the thing that eventually saved the afternoon was a log line showing it had chosen delete_file over delete_artifact on every single step. That log turned "the agent is broken" into "the descriptions overlap," which is a five-minute fix instead of an all-day mystery. A router that decides silently is a router you'll debug blind; a router that narrates its decisions hands you the answer before you have to go looking for it.

The descriptions and schemas that make routing reliable are worth treating as first-class assets — they're the difference between an agent that deletes artifacts and one that deletes your source. Keeping your sharpest, least-ambiguous tool descriptions in a prompt library like PromptABCD, tagged by the harness they route within, means you stop rewriting them from memory and start reusing the versions you already proved the model can route correctly. The router is code; the descriptions are the real interface. Version them like it.