Agent Tool Routing Inside an AI Harness

Q: Common Mistakes

⚠️ **Common mistake:** Registering two tools whose descriptions overlap and then blaming the model when it picks wrong.

An agent I was debugging last year kept deleting the wrong files. The task was "clean up stale build artifacts." Instead, it ran

delete

on source files, over and over, cheerfully reporting success. The model wasn't broken. The prompt was fine. The failure was in agent tool routing — the harness had two tools,

delete_artifact

and

delete_file

, with nearly identical descriptions, and the model kept picking the general one. One overlapping sentence in a tool description cost an afternoon and a git reflog rescue.

That's the thing about routing: when it fails, it fails silently and confidently. Let's make it fail loudly instead.

What Is Agent Tool Routing?

Agent tool routing is the step where the harness takes the model's requested action and maps it to an actual function to run. The model says "I want to call

search

with these arguments"; the router decides which

search

that is, whether it exists, whether it's allowed right now, and what to do if any of those checks fail.

In the simplest harness, routing is a dictionary lookup:

hljs python

[object Object], ,[object Object],(,[object Object],):
    fn = tools.get(call.name)
    ,[object Object], fn ,[object Object], ,[object Object],:
        ,[object Object], ,[object Object],
    ,[object Object], fn(**call.args)

What this does: it finds the requested tool by name and runs it, or returns an error if the name doesn't exist. This is correct and complete for a three-tool agent. It's also where every routing bug you'll ever hit starts, because the dictionary lookup is the easy 20% — the other 80% is everything the lookup doesn't check.

Why Routing Deserves More Attention Than It Gets

Most people assume routing failures mean "the model picked the wrong tool." In practice, that's the rare case. The common failures are subtler:

Right tool, wrong arguments. The model picks
```
send_email
```
correctly but passes
```
to="the customer"
```
instead of an address. The lookup succeeds; the call fails downstream.
Overlapping descriptions. Two tools whose descriptions share phrasing — like my
```
delete_artifact
```
and
```
delete_file
```
— and the model can't reliably tell them apart. The router runs exactly what it was told; the ambiguity was upstream.
Tool available at the wrong time. The model calls
```
deploy
```
before
```
run_tests
```
has passed. The tool exists, so the router runs it, and now broken code is in production.

Notice that a plain dictionary lookup catches none of these. It only checks existence. The tool it ran was real, so it reported success — while doing the wrong thing. That gap between "the tool exists" and "the tool should run, with these arguments, right now" is where agent tool routing earns real engineering attention.

⚡ Pro tip: When an agent misbehaves, check your tool descriptions before you touch the routing code. Ambiguous behavior almost always traces to two descriptions that overlap or one description that under-specifies. The router is usually doing its job correctly; the job was just described badly.

How Does Agent Tool Routing Work Beyond a Dictionary?

A production router does three jobs the lookup skips: it validates arguments, it detects ambiguity before it bites, and it gates tools by state. Here's a router that does all three:

hljs python

[object Object], ,[object Object],(,[object Object],):
    spec = tools.get(call.name)
    ,[object Object], spec ,[object Object], ,[object Object],:
        ,[object Object], ,[object Object],

    ,[object Object],
    ,[object Object], call.name ,[object Object], spec.requires ,[object Object], ,[object Object], state.get(spec.requires[call.name]):
        ,[object Object], (,[object Object],
                ,[object Object],)

    ,[object Object],
    ,[object Object],:
        args = schemas[call.name].model_validate(call.args)
    ,[object Object], Exception ,[object Object], e:
        ,[object Object], ,[object Object],

    ,[object Object],
    ,[object Object], spec.fn(**args.model_dump())

What this does: before running anything, it checks whether the tool is permitted in the current state (so

deploy

can't fire until

tests_passed

is true), validates the arguments against a schema, and only then executes. Each guard returns a specific, model-readable error instead of running the wrong thing successfully.

State-gating is the part almost no tutorial covers, and it's the one that prevents the scariest failures. By attaching a

requires

map to each tool —

deploy

requires

tests_passed

refund

requires

manager_approved

— you encode your workflow's real ordering into the router itself. The model can ask for

deploy

early; the router simply won't let it, and tells it why. That's a guardrail the prompt alone can't reliably enforce, because prompts are suggestions and routers are law.

⚡ Pro tip: Measure description overlap before you ship. A quick embedding-similarity check across your tool descriptions surfaces the pairs the model will confuse. If two descriptions score above ~0.85 similarity, rewrite one to be sharply distinct — lead with the difference, not the shared purpose. "Deletes build outputs in ./dist only" beats "Deletes files you don't need."

Naming and Namespacing Tools

The cheapest routing improvement is naming discipline, and it's underrated. Tools that share a prefix and a clear scope are easier for both the model and you to route correctly:

fs.read        fs.write       fs.list
db.query       db.migrate
net.fetch      net.post

What this does: the namespace signals category at a glance, so the model groups related actions and you can apply category-wide rules — like "everything under

db.

requires a

readonly=False

flag to write." Namespacing also makes state-gating readable:

deploy.release requires ci.passed

documents your pipeline in the tool table itself.

Three teams where careful routing changed outcomes:

A DevOps engineer state-gates
```
rollback
```
behind a captured healthy snapshot, so the agent can't roll back to a state that was never known-good.
A healthcare software developer namespaces
```
phi.read
```
separately from
```
public.read
```
and routes every
```
phi.
```
call through an extra audit-logging wrapper, satisfying a compliance requirement in the router rather than in every tool.
A game studio's build engineer adds argument validation to
```
upload_build
```
so the model can't pass a platform string the store doesn't recognize — the router rejects it with the list of valid platforms, and the model self-corrects.

Routing When the Model Asks for Nothing

There's a routing case that isn't about picking between tools — it's the model returning plain text when you expected a tool call, or calling no tool at all mid-task. A naive harness treats "no tool call" as "done" and exits, sometimes leaving the task half-finished.

A sturdier router distinguishes finished from stuck:

hljs python

[object Object], ,[object Object],(,[object Object],):
    ,[object Object], reply.tool_calls:
        ,[object Object], ,[object Object],
    ,[object Object], state[,[object Object],]:
        ,[object Object], ,[object Object],
    ,[object Object],
    ,[object Object], ,[object Object],  ,[object Object],

What this does: it treats an empty tool-call list as three different situations depending on state — act on requested tools, exit only if completion was explicitly signalled, and otherwise nudge the model back to work instead of silently ending a half-done task. That

nudge

branch catches the quiet drop-outs where a model trails off mid-plan.

The nudge should be specific, not scolding. "You stopped after reading the file but haven't written the summary yet — call

write_summary

finish()

" works far better than a generic "continue." You're routing the model back to the missing step, and naming the step is what makes the re-prompt land.

⚡ Pro tip: Count how often your agent hits the

nudge

branch. A high nudge rate usually means your

finish()

tool is missing or badly described — the model wants to stop but has no clean way to say so, so it just goes quiet. Give it an explicit, well-described exit and the silent drop-outs mostly vanish.

Common Mistakes

⚠️ Common mistake: Registering two tools whose descriptions overlap and then blaming the model when it picks wrong. The model routes on the descriptions you give it. If

delete_file

and

delete_artifact

read almost the same, the model's confusion is a specification bug, not a reasoning failure. Either merge the tools and disambiguate with an argument, or rewrite the descriptions so the boundary between them is unmistakable.

A few more routing traps worth naming:

No argument validation. Existence checks aren't enough. A tool that runs with garbage arguments fails in ways that are far harder to trace than a clean
```
ARG ERROR
```
at the router.
Ignoring state. Without state-gating, ordering lives only in the prompt, and prompts don't reliably enforce ordering. Encode "X before Y" in the router where it's actually binding.
Silent success on the wrong tool. The worst outcome isn't an error — it's a successful call that did something you didn't want. Make your router prefer a loud rejection over a quiet mistake every time.

Conclusion

Agent tool routing is more than a dictionary lookup, even though the lookup is where it starts. The real work is in the guards around it: validating arguments so bad calls fail clean, checking descriptions so the model can tell tools apart, and gating tools by state so your workflow's ordering is enforced by code instead of hoped for in a prompt. Get those three right and a whole category of confident, silent failures disappears.

The habit that ties it all together is making the router observable. Every routing decision — which tool, which arguments, which guard passed or blocked it — should land in a log you can read after the fact. When my agent deleted the wrong files, the thing that eventually saved the afternoon was a log line showing it had chosen

delete_file

over

delete_artifact

on every single step. That log turned "the agent is broken" into "the descriptions overlap," which is a five-minute fix instead of an all-day mystery. A router that decides silently is a router you'll debug blind; a router that narrates its decisions hands you the answer before you have to go looking for it.

The descriptions and schemas that make routing reliable are worth treating as first-class assets — they're the difference between an agent that deletes artifacts and one that deletes your source. Keeping your sharpest, least-ambiguous tool descriptions in a prompt library like PromptABCD, tagged by the harness they route within, means you stop rewriting them from memory and start reusing the versions you already proved the model can route correctly. The router is code; the descriptions are the real interface. Version them like it.

An agent I was debugging last year kept deleting the wrong files. The task was "clean up stale build artifacts." Instead, it ran

delete

on source files, over and over, cheerfully reporting success. The model wasn't broken. The prompt was fine. The failure was in agent tool routing — the harness had two tools,

delete_artifact

and

delete_file

, with nearly identical descriptions, and the model kept picking the general one. One overlapping sentence in a tool description cost an afternoon and a git reflog rescue.

That's the thing about routing: when it fails, it fails silently and confidently. Let's make it fail loudly instead.

What Is Agent Tool Routing?

Agent tool routing is the step where the harness takes the model's requested action and maps it to an actual function to run. The model says "I want to call

search

with these arguments"; the router decides which

search

that is, whether it exists, whether it's allowed right now, and what to do if any of those checks fail.

In the simplest harness, routing is a dictionary lookup:

hljs python

[object Object], ,[object Object],(,[object Object],):
    fn = tools.get(call.name)
    ,[object Object], fn ,[object Object], ,[object Object],:
        ,[object Object], ,[object Object],
    ,[object Object], fn(**call.args)

Why Routing Deserves More Attention Than It Gets

Most people assume routing failures mean "the model picked the wrong tool." In practice, that's the rare case. The common failures are subtler:

Right tool, wrong arguments. The model picks
```
send_email
```
correctly but passes
```
to="the customer"
```
instead of an address. The lookup succeeds; the call fails downstream.
Overlapping descriptions. Two tools whose descriptions share phrasing — like my
```
delete_artifact
```
and
```
delete_file
```
— and the model can't reliably tell them apart. The router runs exactly what it was told; the ambiguity was upstream.
Tool available at the wrong time. The model calls
```
deploy
```
before
```
run_tests
```
has passed. The tool exists, so the router runs it, and now broken code is in production.

How Does Agent Tool Routing Work Beyond a Dictionary?

A production router does three jobs the lookup skips: it validates arguments, it detects ambiguity before it bites, and it gates tools by state. Here's a router that does all three:

hljs python

[object Object], ,[object Object],(,[object Object],):
    spec = tools.get(call.name)
    ,[object Object], spec ,[object Object], ,[object Object],:
        ,[object Object], ,[object Object],

    ,[object Object],
    ,[object Object], call.name ,[object Object], spec.requires ,[object Object], ,[object Object], state.get(spec.requires[call.name]):
        ,[object Object], (,[object Object],
                ,[object Object],)

    ,[object Object],
    ,[object Object],:
        args = schemas[call.name].model_validate(call.args)
    ,[object Object], Exception ,[object Object], e:
        ,[object Object], ,[object Object],

    ,[object Object],
    ,[object Object], spec.fn(**args.model_dump())

What this does: before running anything, it checks whether the tool is permitted in the current state (so

deploy

can't fire until

tests_passed

is true), validates the arguments against a schema, and only then executes. Each guard returns a specific, model-readable error instead of running the wrong thing successfully.

State-gating is the part almost no tutorial covers, and it's the one that prevents the scariest failures. By attaching a

requires

map to each tool —

deploy

requires

tests_passed

refund

requires

manager_approved

— you encode your workflow's real ordering into the router itself. The model can ask for

deploy

early; the router simply won't let it, and tells it why. That's a guardrail the prompt alone can't reliably enforce, because prompts are suggestions and routers are law.

Naming and Namespacing Tools

The cheapest routing improvement is naming discipline, and it's underrated. Tools that share a prefix and a clear scope are easier for both the model and you to route correctly:

fs.read        fs.write       fs.list
db.query       db.migrate
net.fetch      net.post

What this does: the namespace signals category at a glance, so the model groups related actions and you can apply category-wide rules — like "everything under

db.

requires a

readonly=False

flag to write." Namespacing also makes state-gating readable:

deploy.release requires ci.passed

documents your pipeline in the tool table itself.

Three teams where careful routing changed outcomes:

A DevOps engineer state-gates
```
rollback
```
behind a captured healthy snapshot, so the agent can't roll back to a state that was never known-good.
A healthcare software developer namespaces
```
phi.read
```
separately from
```
public.read
```
and routes every
```
phi.
```
call through an extra audit-logging wrapper, satisfying a compliance requirement in the router rather than in every tool.
A game studio's build engineer adds argument validation to
```
upload_build
```
so the model can't pass a platform string the store doesn't recognize — the router rejects it with the list of valid platforms, and the model self-corrects.

Routing When the Model Asks for Nothing

A sturdier router distinguishes finished from stuck:

hljs python

[object Object], ,[object Object],(,[object Object],):
    ,[object Object], reply.tool_calls:
        ,[object Object], ,[object Object],
    ,[object Object], state[,[object Object],]:
        ,[object Object], ,[object Object],
    ,[object Object],
    ,[object Object], ,[object Object],  ,[object Object],

nudge

branch catches the quiet drop-outs where a model trails off mid-plan.

The nudge should be specific, not scolding. "You stopped after reading the file but haven't written the summary yet — call

write_summary

finish()

" works far better than a generic "continue." You're routing the model back to the missing step, and naming the step is what makes the re-prompt land.

⚡ Pro tip: Count how often your agent hits the

nudge

branch. A high nudge rate usually means your

finish()

tool is missing or badly described — the model wants to stop but has no clean way to say so, so it just goes quiet. Give it an explicit, well-described exit and the silent drop-outs mostly vanish.

Common Mistakes

⚠️ Common mistake: Registering two tools whose descriptions overlap and then blaming the model when it picks wrong. The model routes on the descriptions you give it. If

delete_file

and

delete_artifact

A few more routing traps worth naming:

No argument validation. Existence checks aren't enough. A tool that runs with garbage arguments fails in ways that are far harder to trace than a clean
```
ARG ERROR
```
at the router.
Ignoring state. Without state-gating, ordering lives only in the prompt, and prompts don't reliably enforce ordering. Encode "X before Y" in the router where it's actually binding.
Silent success on the wrong tool. The worst outcome isn't an error — it's a successful call that did something you didn't want. Make your router prefer a loud rejection over a quiet mistake every time.

Conclusion

delete_file

over

delete_artifact

Tool Routing Inside an AI Harness: A Practical Guide

What Is Agent Tool Routing?

Why Routing Deserves More Attention Than It Gets

How Does Agent Tool Routing Work Beyond a Dictionary?

Naming and Namespacing Tools

Routing When the Model Asks for Nothing

Common Mistakes

Conclusion

Continue Reading

The Parsing Layer: Turning Model Output Into Actions

Building a Minimal Agent Harness in Python From Scratch

Harness vs Framework: What's the Difference for AI Agents?

Tool Routing Inside an AI Harness: A Practical Guide

What Is Agent Tool Routing?

Why Routing Deserves More Attention Than It Gets

How Does Agent Tool Routing Work Beyond a Dictionary?

Naming and Namespacing Tools

Routing When the Model Asks for Nothing

Common Mistakes

Conclusion

Continue Reading

The Parsing Layer: Turning Model Output Into Actions

Building a Minimal Agent Harness in Python From Scratch

Harness vs Framework: What's the Difference for AI Agents?