Dynamic Tool Selection Within the Loop

An agent I reviewed had access to 40 tools. On any given step it picked the wrong tool about 30% of the time — reaching for

update_record

create_record

The fix is a dynamic tool selection agent: instead of showing the model every tool on every step, you narrow the available tools to the handful that fit the current situation. Tool accuracy on that agent jumped from 70% to 94% after the change, with no model upgrade. This post explains how dynamic tool selection works and how to add it to your loop.

What Is Dynamic Tool Selection?

Dynamic tool selection means the set of tools offered to the model changes based on context, rather than being a fixed list baked into the system prompt. On each loop step, a selection layer decides which tools are relevant right now and exposes only those.

Think of it as the difference between a mechanic's entire garage and the specific tray of tools they carry to a job. The garage has everything, but you hand them the tray that matches the task. The model reasons far better over five well-chosen tools than over forty, because every irrelevant tool is a distractor that adds a chance of a wrong pick.

This matters more as your agent grows. With five tools, a static list is fine. Past roughly fifteen, selection accuracy starts sliding, descriptions blur together, and the prompt balloons. A dynamic tool selection agent scales past that ceiling because the model never sees the full catalog at once.

Why Static Tool Lists Break Down

Two forces degrade a big static toolset. The first is prompt dilution: each tool's description eats tokens and attention, and forty descriptions bury the three that matter for the current step. The second is semantic collision — the more tools you add, the more their descriptions overlap, and the model's choice between

get_user

fetch_customer

There's a cost angle too. Every tool definition is tokens sent on every single step. An agent with 40 verbose tool schemas might spend 3,000 tokens per step just describing tools it won't use. Across a long loop, that's most of your bill going to a menu the model mostly ignores.

⚡ Pro tip: Measure your tool-selection accuracy directly. Log the tool the model picked and, on a sample, have a human or a stronger model judge whether it was the right pick. If accuracy is under 90% and you have more than fifteen tools, dynamic selection will help more than any prompt tweak.

How to Build a Dynamic Tool Selection Agent

The core is a selection function that runs before each model call and returns a relevant subset. The simplest effective version uses retrieval: embed the tool descriptions once, embed the current task state, and pull the top-K nearest tools.

[object Object], ,[object Object],(,[object Object],):
    query = summarize_current_need(state)   ,[object Object],
    q_vec = embed(query)
    scored = [(cosine(q_vec, t.embedding), t) ,[object Object], t ,[object Object], all_tools]
    scored.sort(reverse=,[object Object],)
    chosen = [t ,[object Object], _, t ,[object Object], scored[:k]]
    ,[object Object], chosen + [finish_tool]           ,[object Object],

What this does: It turns the agent's current need into a vector, ranks all tools by similarity, and returns the top few plus the always-available finish action — so the model sees a short, relevant menu instead of the whole catalog.

Wire it into the loop so selection happens fresh each step, because the right toolset for step one (search) differs from step five (write results).

[object Object], ,[object Object],(,[object Object],):
    ,[object Object], step ,[object Object], ,[object Object],(max_steps):
        tools = select_tools(state, all_tools, k=,[object Object],)
        action = model_decide(state, tools)
        ,[object Object], action.name == ,[object Object],:
            ,[object Object], action.args[,[object Object],]
        state = update_state(state, action, run_tool(action))
    ,[object Object], force_answer(state)

What this does: It recomputes the relevant toolset on every iteration, so the model's options track the evolving task instead of being fixed at the start — the essence of a dynamic tool selection agent.

⚡ Pro tip: Always include a small set of "core" tools in every selection regardless of similarity — usually

finish

Grouping Tools for Better Selection

Pure retrieval works, but grouping tools into capability clusters often works better and is easier to reason about. Define named toolsets — "search," "write," "reporting" — and select which cluster is active, then expose that cluster's tools.

TOOLSETS = {
    ,[object Object],: [search_docs, get_record, list_items],
    ,[object Object],: [create_record, update_record, delete_record],
    ,[object Object],: [build_chart, export_csv, email_summary],
}

,[object Object], ,[object Object],(,[object Object],):
    intent = classify_intent(state)   ,[object Object],
    ,[object Object], TOOLSETS.get(intent, TOOLSETS[,[object Object],]) + [finish_tool]

What this does: It classifies the current intent (reading, writing, or reporting) and exposes only that cluster of tools, which sharply reduces cross-category confusion like calling a write tool during a read phase.

The tradeoff between retrieval and grouping: retrieval handles fuzzy, overlapping needs and scales to hundreds of tools; grouping is more predictable and debuggable but needs you to maintain the clusters. For most teams under a hundred tools, I actually prefer grouping — it's easier to explain why a tool was or wasn't offered, which matters a lot when you're debugging a wrong pick at 2am.

The two approaches also fail differently, which is worth knowing before you pick. Retrieval fails silently: when the embedding of your task doesn't quite match the embedding of the right tool, that tool just never appears, and nothing tells you it was missing. Grouping fails loudly: when your intent classifier routes to the wrong cluster, the right tool is absent and the model visibly flails among tools that don't fit. Loud failures are annoying but easy to catch and fix; silent ones can hide in your metrics for weeks. If you go the retrieval route, log the rank of the eventually-correct tool so you can spot the near-misses that almost dropped out of the top-K.

When Does an Agent Have Too Many Tools?

The honest answer is that it depends on how similar the tools are, not just how many there are. Ten tools that do obviously different things — search, send email, run SQL, resize an image — rarely confuse a capable model, because the descriptions don't overlap. Ten tools that all touch customer records with subtly different semantics will confuse it at five, let alone ten.

So the real threshold is semantic density, and you can estimate it cheaply. Embed every tool description and compute the average pairwise similarity. A toolset where the closest pairs sit far apart in vector space tolerates a longer static list. A toolset where several tools cluster tightly needs dynamic selection much sooner, because those clusters are exactly where the model's picks blur. I've seen a twelve-tool agent that badly needed selection and a thirty-tool agent that was fine — the difference was entirely how distinguishable the tools were from each other.

A related signal lives in your logs: track which tools get confused for which. If you plot a confusion matrix of "tool the model should have picked" against "tool it actually picked," the hot cells almost always involve near-duplicate descriptions. Those pairs are your first candidates either to merge, to rename for clarity, or to split across different selection groups so they never appear together. Sometimes the fix isn't selection at all — it's rewriting two muddy descriptions so the model can finally tell them apart.

⚡ Pro tip: Before building a selection layer, spend an hour rewriting your tool descriptions to emphasize what makes each one distinct from its nearest neighbor. Lead each description with when to use it versus the similar tool, not just what it does. For agents under fifteen tools, this often recovers most of the accuracy you'd otherwise chase with retrieval — and it makes any selection layer you add later work better too.

Common Mistakes

⚠️ Common mistake: Setting K too low to save tokens. If your selector returns only three tools and the right one ranks fourth, the model literally cannot pick it — you've converted a selection problem into an impossibility. Start with K around six to eight, measure how often the correct tool falls outside the top-K, and only tighten once that miss rate is near zero.

A second mistake is selecting tools from a stale query. If you build the selection query from the original user message and never update it, by step six you're choosing tools for step one's needs. Rebuild the query from current state each step.

A third is forgetting that selection itself can be wrong. When accuracy is critical, log which tools were offered alongside which was picked. Sometimes the model chose well from a bad menu — that's a selector bug, not a model bug, and you'll never find it if you only log the final pick.

Consider three teams here. A CRM automation agent at a SaaS company grouped its 30 tools into read/write/report clusters and cut wrong-tool errors by two-thirds. A DevOps agent used retrieval over 120 infrastructure tools and kept per-step token cost flat as the catalog grew. And a content-ops agent for a media team combined both — grouping first, then retrieval within the group — to stay accurate across 60 publishing and analytics tools.

Conclusion

A dynamic tool selection agent solves the problem that big static toolsets create: too many similar options degrade the model's choices and inflate cost. Whether you narrow by retrieval, by capability groups, or both, the principle holds — show the model the few tools that fit the moment, not the whole garage.

Selection logic and cluster definitions are exactly the kind of building blocks worth keeping reusable. A prompt and snippet library like PromptABCD lets you store your selection prompts, intent-classifier wording, and toolset groupings in one versioned place, so the next agent you build gets sharp tool selection without you re-deriving it from scratch.