Managing the Context Window Across Loop Iterations

Why does your agent get slower, pricier, and noticeably dumber the longer it runs? It starts sharp at step two and by step twelve it's forgetting instructions, repeating itself, and taking a full second longer per call. The culprit is almost always the agent loop context window quietly filling with junk — every raw tool result, every verbose thought, every stack trace, all re-sent on every single turn. The model isn't degrading. Its input is.

An agent's context window carries the entire transcript back to the model each iteration. Left unmanaged, that transcript grows without bound: step ten re-sends everything from steps one through nine, most of it no longer relevant. More tokens mean higher cost, higher latency, and — past a point — worse reasoning, because the signal the model needs is buried under accumulated noise. Let's tear a bloated loop apart.

Before: The Weak Prompt

Here's the loop, or rather the loop's context handling, in its naive form:

messages = [system, user_goal]
,[object Object], step ,[object Object], ,[object Object],(max_steps):
    reply = model.call(messages)          ,[object Object],
    messages.append(reply.as_message())
    ,[object Object], call ,[object Object], reply.tool_calls:
        raw = tools[call.name](**call.args)
        messages.append(tool_result(call.,[object Object],, ,[object Object],(raw)))   ,[object Object],

What this does: appends every model reply and every raw tool result to a single ever-growing list that's re-sent in full on every call — so by late iterations the model is re-reading a mountain of stale detail each turn.

Run this for fifteen steps against tools that return large payloads and the final call carries tens of thousands of tokens, most of them results the agent already used and no longer needs.

And the growth isn't linear in the way you'd hope — it's closer to quadratic in effort. Each new turn adds to the transcript, and the whole fattened transcript is re-sent on the next turn, so you're not just paying for the new content, you're re-paying for all the old content every single step. A twenty-step run doesn't cost twenty units; it costs something closer to the sum of a growing series. That compounding is why an agent that felt cheap in testing on short tasks becomes alarming on the first long one — the cost curve was hiding in the steps you didn't run.

Why It Fails

The window fills with three kinds of junk, and each is avoidable.

First, raw tool payloads — a search tool returns a hundred results, the agent needed three, and the other ninety-seven ride along in context for the rest of the run. Second, verbose intermediate reasoning — every "let me think about this" thought preserved verbatim forever, though only its conclusion matters later. Third, resolved detours — a failed attempt, its error, and the recovery all kept in full even though only the outcome is relevant going forward.

The result isn't just cost and latency. Past a certain fill level, models genuinely reason worse — the instruction from step one gets diluted, and the model's attention scatters across pages of stale detail. A full context window degrades quality, not only economics. The teardown fix is to treat the window as a managed resource, not an append-only log.

This is the part that surprises people: managing the agent loop context window isn't primarily a cost optimization, it's a quality one. Teams reach for it to save money and discover the bigger win is accuracy — a lean window keeps the model's attention on the goal and the recent observations instead of forcing it to re-parse ninety-seven search results it already discarded. The savings are real, but the sharper reasoning is what actually moves your success rate.

⚠️ Common mistake: Assuming a bigger context window solves this. A larger window delays the wall but doesn't move it — you just pay to re-send more junk before quality drops, and it drops later but just as hard. The fix isn't more room for noise; it's less noise. Manage what goes in rather than buying space to ignore the problem. A bigger window is a bigger desk; it still gets buried if you never file anything away.

⚡ Pro tip: Log your context token count per iteration and plot it. A healthy managed loop stays roughly flat or grows slowly; a bloated one climbs linearly. That single line tells you instantly whether your loop manages context or just accumulates it.

After: The Improved Prompt

The fix is to curate context each turn: keep what future steps need, compress or drop what they don't.

[object Object], ,[object Object],(,[object Object],):
    system, goal = messages[,[object Object],], messages[,[object Object],]
    recent = messages[-keep_recent:]
    old = messages[,[object Object],:-keep_recent]
    summary = summarize_old(old) ,[object Object], old ,[object Object], ,[object Object],       ,[object Object],
    kept = [system, goal]
    ,[object Object], summary:
        kept.append(assistant_note(,[object Object],))
    ,[object Object], kept + recent

What this does: preserves the system prompt, the goal, and the most recent turns verbatim, while compressing the older middle of the transcript into a short progress summary — keeping continuity without re-sending every stale detail.

[object Object], ,[object Object],(,[object Object],):
    trimmed = extract_relevant(raw, max_items=,[object Object],)         ,[object Object],
    messages.append(tool_result(call.,[object Object],, trimmed))
    archive[call.,[object Object],] = raw                               ,[object Object],

What this does: puts only the relevant slice of a tool result into context while archiving the full payload elsewhere, so the agent can fetch details on demand without carrying the whole payload every turn.

Breaking Down Each Element

Three moves keep the window lean.

Trimming at the source — storing only the relevant slice of each tool result — stops junk from entering the window in the first place, which beats cleaning it up later. Summarizing the middle — compressing older turns into a progress note — preserves continuity without the verbatim bulk. And archiving off-context — keeping full payloads retrievable but out of the window — means nothing is lost, only moved, so the agent can pull detail back if it genuinely needs it.

Together they hold the window roughly flat as the run grows. The agent keeps its recent working memory and a summary of its history, instead of dragging every raw byte forward forever.

The mental model that helps: treat the agent loop context window like a desk, not an archive. A desk holds what you're working on right now plus a note of where things stand; everything else goes in a drawer you can open when needed. An agent whose desk is buried under every document it has ever touched works exactly as badly as a person in the same situation — slowly, and with things constantly falling through the cracks. Curation keeps the desk clear.

⚡ Pro tip: Keep the most recent few turns verbatim and only summarize what's older. Recency matters — the agent's immediate next step depends on the last couple of observations in full detail, while step three's raw output can safely become one line of summary.

Variations for Different Contexts

Context strategy shifts with the work.

A research analyst's long-running agent leans hard on summarization, collapsing each explored source into a one-line finding and archiving the full text. A software engineer's coding agent keeps recent file states verbatim but summarizes older edits, since the current code matters far more than the history of how it got there. A customer-service agent handling long conversations keeps a running summary of resolved issues and only the active thread in full, so a forty-message chat stays as lean as a five-message one.

Same window, three curation policies — each keeping what its task needs live and compressing the rest.

⚡ Pro tip: Give the agent a

recall(id)

⚡ Pro tip: Trigger summarization by token threshold, not step count. A step that returns a huge payload should trigger compaction immediately, while ten tiny steps might not need it yet. Watch the actual fill level and compress when it crosses a line, not on a fixed schedule.

Save and Reuse This

Context management isn't task-specific plumbing you rebuild each time — it's the same trim-summarize-archive pattern with the knobs set differently. Swap what counts as "relevant," tune how much recent history stays verbatim, and the machinery carries over unchanged. The policy is task-specific; the plumbing is not. That split is exactly why it's worth building once properly: the hard part — trimming safely, summarizing without losing thread, archiving retrievably — is identical across agents, and only the "what counts as relevant" knob changes from one to the next.

I keep the

trim_context

store_result