How to Cache Tool Results Across Iterations

Picture this: you're watching your agent's trace and you notice it calling the same weather API four times in a single run — same city, same day, four identical requests, four identical responses, four times the cost and latency. The agent doesn't remember it already asked, so it asks again every time its reasoning circles back. Agent loop tool caching fixes this by remembering tool results within and across runs, so the agent pays for each distinct lookup once.

This post covers how to add caching to an agent loop that cuts redundant calls dramatically — without the classic trap of serving stale data that makes the agent confidently wrong.

What Is Tool Result Caching in an Agent Loop?

Agent loop tool caching means storing the results of tool calls so that a repeated call with the same arguments returns the stored result instead of hitting the tool again. When the agent calls get_weather("Austin") and later calls it again in the same run, the second call returns the cached response instantly and for free.

The cache lives at the loop level, wrapping your tool execution. Every tool call is keyed by its name and arguments; before running the tool, the loop checks whether it already has a result for that key. If it does, it returns the cached value; if not, it runs the tool and stores the result. Simple in concept, and the details are where it gets interesting.

There are two scopes worth distinguishing. Within-run caching stops an agent from re-fetching the same data during a single loop — the four-weather-calls problem. Cross-run caching shares results across different runs and users, so a lookup one user triggered can serve another. Within-run caching is almost always safe and beneficial; cross-run caching is powerful but demands more care about freshness and isolation.

Why It Matters

Redundant tool calls are one of the quietest sources of agent cost and latency. They don't show up as errors — the agent works fine, it's just paying two, three, four times for data it already has. On an agent making expensive API calls or slow database queries, deduplicating those repeats can cut cost and latency substantially with zero effect on answer quality.

The within-run case is pure upside. There is no good reason for an agent to fetch identical data twice in one reasoning session; the second fetch is waste by definition. Caching it away is free performance. This alone justifies adding a cache to almost any loop.

Cross-run caching unlocks bigger savings on agents with overlapping traffic. If a hundred users ask about the same popular product, caching the product lookup means you hit the backend once instead of a hundred times. But now freshness matters — a cached price that's an hour stale might be wrong — so cross-run caching is a real engineering decision, not a free win.

The cost savings compound in a way that's easy to underestimate. Because agent loops re-send context every step, a redundant tool call isn't just the wasted API cost — it's also the tokens spent processing that redundant result on every subsequent step. A duplicate fetch early in a long loop gets re-read a dozen times before the run ends. Eliminating it with agent loop tool caching saves the fetch and all the downstream token cost of carrying its redundant result forward. On long loops with expensive tools, this second-order saving often exceeds the first.

⚡ Pro tip: Instrument your cache hit rate from day one, broken down per tool. A tool with a 60% hit rate is saving you real money and is clearly worth caching; a tool with a 2% hit rate is adding complexity for almost no benefit and probably shouldn't be cached at all. The hit rate tells you exactly which tools deserve a cache and which are just cache-key overhead pretending to help.

⚡ Pro tip: Start with within-run caching only. It captures a surprising share of the waste — agents circle back to the same data constantly within a run — and it carries almost no freshness risk, because a single run is short enough that data rarely changes underneath it. Add cross-run caching later, deliberately, only for tools whose data is stable enough to share.

How to Build a Tool Cache for Your Loop

The core is a cache keyed by the tool call's identity, wrapping your tool executor. Here's a within-run cache, the safe default.

[object Object], ,[object Object],(,[object Object],):
    key = (action.name, ,[object Object],(,[object Object],(action.args.items())))
    ,[object Object], key ,[object Object], cache:
        ,[object Object], cache[key]                 ,[object Object],
    result = run_tool(action)
    cache[key] = result
    ,[object Object], result

,[object Object], ,[object Object],(,[object Object],):
    cache = {}                            ,[object Object],
    ,[object Object], _ ,[object Object], ,[object Object],(max_steps):
        action = model_decide(state, tools)
        ,[object Object], action.name == ,[object Object],:
            ,[object Object], action.args[,[object Object],]
        result = cached_run_tool(action, cache)
        state = update_state(state, action, result)
    ,[object Object], force_answer(state)

What this does: It keys each tool call by its name and sorted arguments, returns a stored result when the same call repeats within the run, and otherwise runs the tool and caches it — eliminating the duplicate-fetch waste with a fresh cache per run so nothing leaks between runs.

For cross-run caching, you add a shared store and a time-to-live so entries expire before they go dangerously stale.

[object Object], time

,[object Object], ,[object Object],(,[object Object],):
    key = (action.name, ,[object Object],(,[object Object],(action.args.items())))
    entry = store.get(key)
    ,[object Object], entry ,[object Object], (time.monotonic() - entry[,[object Object],]) < ttl_s:
        ,[object Object], entry[,[object Object],]            ,[object Object],
    result = run_tool(action)
    store[key] = {,[object Object],: result, ,[object Object],: time.monotonic()}
    ,[object Object], result

What this does: It shares cached results across runs but attaches a time-to-live, so an entry is reused only while it's fresh and re-fetched once it expires — trading a bounded amount of staleness for a large drop in backend load on popular lookups.

Which Tools Should and Shouldn't Be Cached

Not every tool belongs in a cache, and caching the wrong one causes real bugs. Cache tools that are read-only and whose results are stable over your caching window — reference data, documentation lookups, configuration, historical facts. These are safe and high-value.

Never cache tools with side effects. A send_email or create_record tool must run every time it's called; caching it would mean the second "call" silently does nothing, which is a serious bug. Your cache should only ever wrap read operations, and it's worth enforcing that in code by marking tools as cacheable explicitly rather than caching everything by default.

Be cautious with tools whose data changes fast. A stock price, an inventory count, a live status — caching these even briefly can serve wrong data. Either don't cache them, or use a very short time-to-live and accept the small staleness explicitly.

⚡ Pro tip: Mark cacheability as a property of each tool, not a blanket loop setting. Tag each tool cacheable: true/false and, for cacheable ones, its acceptable staleness. Then the cache layer reads those tags instead of guessing. This makes the safe/unsafe decision explicit and per-tool, which is exactly where it belongs — a global "cache everything" switch is how side-effecting tools end up silently skipped.

Common Mistakes

⚠️ Common mistake: Caching a tool with side effects. If you wrap your whole tool executor in a cache without excluding writes, the second call to create_record with the same arguments returns the cached "success" without actually creating the record — and the agent believes it did the work. This is a genuinely dangerous bug because it fails silently. Only cache read-only tools, and enforce that boundary in code, not just convention.

A second mistake is ignoring cache-key collisions from argument formatting. If the agent calls get_user(id=42) once and get_user(id="42") another time, a naive key treats these as different and misses the cache — or worse, a sloppy key treats genuinely different calls as the same. Normalize arguments before keying, so equivalent calls hit and distinct calls miss.

A third is caching across users without isolation. If your cache key doesn't include the user or tenant, one user's private data can be served to another from the cache — a caching bug that's also a security incident. Cross-run caches for user-scoped data must include the identity in the key or stay per-user.

Three teams show the range. A travel-booking agent cached destination and hotel reference data cross-run with a one-hour TTL, cutting API cost by half while leaving live availability uncached. A support agent used within-run caching only and eliminated the duplicate-lookup waste with zero freshness risk. And a data-analytics agent cached expensive aggregate queries per-session, so an analyst iterating on a report didn't re-run the same costly query on every follow-up question.

Conclusion

Agent loop tool caching is one of the cheapest performance wins available: within-run caching kills duplicate fetches for free, and cross-run caching cuts backend load on popular lookups when you handle freshness deliberately. The rules are simple — cache reads, never writes, key carefully, isolate by user, and expire stale entries.

The cache wrapper, the cacheability tagging, and the TTL logic are reusable across every agent you build. A prompt and snippet library like PromptABCD is a handy home for these patterns, so your next loop gets safe, effective caching from the start instead of paying for the same data four times before someone notices.

⚡ Pro tip: When you add cross-run caching, add cache invalidation at the same time, not later. If a write tool changes data that a read tool caches, the write should invalidate the relevant cache entries, or the agent will read its own stale cache right after updating the underlying record. The read-after-write inconsistency is one of the most confusing bugs in a cached agent, and it's trivial to prevent if you wire invalidation in from the start instead of bolting it on after it bites you.