Context Compaction Between Agent Turns

Most advice on agent context compaction is backwards. It tells you to compress the transcript on a fixed schedule — every N turns, summarize everything older — which is exactly how agents lose the one detail they needed three steps later. Compaction isn't about shrinking on a timer; it's about deciding what's still relevant and protecting it while discarding the rest. Get that backwards and you build an agent that reliably forgets the constraint it was supposed to honor.

Context compaction is the act of replacing a chunk of transcript with a shorter version that preserves what future turns need. Done by relevance, it keeps an agent sharp across long runs. Done by the clock, it's a slow way to introduce amnesia. This guide does it by relevance.

Quick-Start (Copy This Right Now)

[object Object], ,[object Object],(,[object Object],):
    ,[object Object], count_tokens(messages) < budget_tokens:
        ,[object Object], messages                       ,[object Object],
    pinned = [m ,[object Object], m ,[object Object], messages ,[object Object], m.pinned]          ,[object Object],
    recent = messages[-,[object Object],:]                              ,[object Object],
    middle = [m ,[object Object], m ,[object Object], messages[,[object Object],:-,[object Object],] ,[object Object], ,[object Object], m.pinned]
    summary = summarize(middle, focus=extract_goal(messages))
    ,[object Object], pinned + [note(summary)] + recent

What this does: compacts only when the context actually exceeds a token budget, protects pinned messages and recent turns, and summarizes the middle with the goal as the focus — so compression is triggered by need and steered by relevance, not by a timer.

Two things make this different from the usual advice: it fires on a token budget, not a turn count, and it summarizes toward the goal rather than producing a neutral recap.

Those two choices sound small and change everything downstream. Firing on a budget means the agent compacts exactly when it's under real memory pressure and never when it isn't — no gratuitous summarization of a short run, no waiting through a bloated one. Summarizing toward the goal means the compressor has a criterion for what to keep: relevance to the remaining work, rather than the summarizer's untethered guess at what's "important." A guess protects nothing in particular; a criterion protects the constraint, the decision, the dead end you can't afford to forget.

Understanding the Variables

Three decisions define good agent context compaction, and each is where the timer-based approach goes wrong.

When to compact should be a token threshold, not a schedule — you compress when the window is genuinely full, not because five turns elapsed. What to protect is the goal, the system prompt, any hard constraints, and the most recent turns; these get pinned and never compacted. And how to summarize should be goal-directed: a summary written to preserve what this task needs, not a generic abstract that treats every sentence as equally worth keeping.

The timer approach gets all three wrong — it fires on time, protects nothing specifically, and summarizes neutrally. That's why it drops constraints: a neutral summary of "the user said keep it under $500" competes with everything else for space and often loses.

Reframing agent context compaction around relevance instead of timing changes what you optimize. A timer-based compactor asks "is it time to shrink?" A relevance-based one asks "what here still steers the remaining work, and what's just history?" The second question is harder — it requires knowing the goal — but it's the only one whose answer protects the details that matter. Compaction that doesn't know the goal is compression with the safety off.

⚡ Pro tip: Pin hard constraints explicitly so compaction can never touch them. Budget limits, must-follow rules, the actual goal — mark these as un-compactable. Most catastrophic compaction failures are a summary that quietly dropped the one rule the whole task depended on.

Step-by-Step: Compacting by Relevance

Build it in three moves.

First, mark what's protected. Tag the goal, constraints, and system prompt as pinned the moment they enter context, so no compaction pass can summarize them away. Second, trigger on budget. Check token count each turn and compact only when it crosses your threshold. Third, summarize with focus — pass the goal into the summarizer so it keeps goal-relevant detail and compresses the rest.

[object Object], ,[object Object],(,[object Object],):
    prompt = (
        ,[object Object],
        ,[object Object],
        ,[object Object],
        ,[object Object],
        ,[object Object],
        ,[object Object],
    )
    ,[object Object], model.call(prompt)

What this does: summarizes the middle of the transcript with explicit instructions to preserve decisions, facts, constraints, and tried-and-failed approaches relevant to the goal — so the compacted version keeps what steers future turns and sheds only what doesn't.

Note the "dead ends already tried" clause. A compaction that forgets what the agent already attempted invites it to repeat those attempts — one of the sneakiest ways bad compaction wastes an entire run.

Equally important is what the summarizer is told to drop: pleasantries, verbose deliberation, the model's own hedging. These make up a surprising fraction of a transcript and carry almost no forward-looking value. Naming them explicitly in the prompt matters, because a summarizer left to its own judgment tends to preserve fluent prose (which is easy to compress well) and cut terse facts (which are hard to phrase smoothly) — exactly backwards from what you want. Tell it to keep the ugly, load-bearing details and shed the smooth, disposable ones.

⚠️ Common mistake: Compacting the recent turns along with the old ones. The agent's next move depends on the last few observations in full fidelity; summarize those and you kick the legs out from under the very step it's about to take. Always keep a verbatim recent window and only compact what's genuinely behind the agent.

Pro-Level Variations

The compaction policy shifts by task.

A legal-research analyst's agent compacts explored documents into one-line holdings but pins every citation verbatim, because a summarized citation is a useless one. A DevOps engineer's incident agent pins the active incident's key facts and compacts the diagnostic history, so the current state stays sharp while the investigation trail compresses. A sales-ops agent working a long account thread pins the deal terms and compacts the small talk, keeping the numbers exact and the pleasantries gone.

Same mechanism, different pins — each task protects the details that would be catastrophic to lose.

⚡ Pro tip: Compact in the background between turns, not in the critical path. If your agent is waiting on a slow tool call anyway, run the compaction pass during that idle time so it adds no latency the user feels. Compaction is a model call; hiding it behind time you're already spending makes it effectively free.

⚡ Pro tip: Have the summarizer output structured fields — decisions, facts, constraints, tried — rather than prose. Structured compaction is far less likely to silently drop a category, because an empty "constraints" field is visibly empty in a way a smooth paragraph never is.

Troubleshooting Common Issues

If your agent repeats work after compaction, your summary is dropping the "already tried" history. Add it explicitly to what the summarizer must preserve.

If your agent violates a constraint only on long runs, the constraint is getting compacted away. Pin it, and confirm the pin survives the compaction pass.

If compaction itself costs too much, you're firing it too often — raise the token threshold so it triggers on genuine pressure, not on every turn once you cross the line once. Compaction is a model call too; it shouldn't run more than it needs to, and running it every turn quietly doubles your call count for no real benefit.

⚡ Pro tip: Keep the pre-compaction transcript archived, not deleted. If a compaction turns out to have dropped something important, you want the original to recover from — and to debug why your summarizer discarded it. Compaction should be reversible in your logs even when it's irreversible in context.

Your Turn

Take a long-running agent and replace any timer-based summarization with the relevance version: pin the goal and constraints, trigger on a token budget, and summarize with the goal in focus and "already tried" preserved. Run it on your longest task and check whether the constraints survive to the end — they usually didn't before.

That check is the whole test. Take the single most important instruction the agent was given, run it long enough to trigger several compactions, and confirm the instruction is still present and precise at the final step. If it survived, your compaction is doing its job; if it blurred or vanished, you've found the bug that was quietly failing your long runs — and now you know exactly which pin to add.

The compaction helper and the goal-directed summary prompt are reusable across every long-running agent. I keep them saved and versioned in PromptABCD, so a new agent compacts by relevance from day one — instead of inheriting the timer-based approach that quietly forgets the one rule the task hinged on.