Scratchpads and Working Memory in the Agent Loop

Two agents, same model, same tools, same task — one solves it reliably and one flails. The only difference: the reliable one has a scratchpad. That gap surprises people who assume capability lives entirely in the model, but it shows up again and again. A place to write things down changes what an agent can do, because agent scratchpad memory gives it something a raw transcript doesn't: a stable, editable workspace that survives the noise of a long run. This is the story of a team that doubled their success rate by adding one.

A scratchpad is a dedicated slice of the agent's context — separate from the conversation transcript — where the agent records intermediate results, running conclusions, and its plan. Unlike the transcript, which just accumulates, the scratchpad is maintained: the agent updates it, corrects it, and reads it back deliberately. It's the difference between a pile of receipts and a running balance.

The Problem the Team Faced

A team built an agent to reconcile invoices against purchase orders — a task with dozens of line items, each needing a match, a check, and a note. Framed as a plain loop, the agent read invoices, called tools, and reasoned turn by turn. But by the twentieth line item it had lost track of which items it had already reconciled, which were flagged, and what the running discrepancy total was. It would re-check items it had cleared and forget flags it had raised.

The information was technically in the transcript — every check the agent had done was somewhere in the history. But it was scattered across twenty turns of raw dialogue, and the model couldn't reliably reconstruct the current state from that sprawl. It had all the pieces and no assembled picture. Success sat around forty percent, and it fell as the invoices got longer.

The Wrong Approach

The team's first fix was to make the agent re-read its history more carefully each turn.

Before each step, carefully review everything you have done so far
in the conversation above, then decide the next action.

What this does: asks the model to reconstruct its current state from the full transcript on every turn — which grows more expensive and less reliable exactly as the task gets longer and the state gets more complex.

It helped a little and scaled terribly. Asking the model to rebuild its state from twenty turns of transcript every step is expensive, slow, and error-prone — the reconstruction itself became a source of mistakes. And it got worse as the task grew, which is precisely backwards: the longer the task, the more state there is to lose and the harder the transcript is to parse. You can't fix a state problem by re-reading the mess that lost the state.

⚠️ Common mistake: Trying to solve a working-memory problem with better recall of the transcript. The transcript is a log, not a workspace — it records what happened in order, not what's currently true. Asking the model to derive current state from a log every turn is asking it to do bookkeeping in its head across pages of dialogue. Give it a workspace instead.

The distinction between a log and a workspace is the whole idea behind agent scratchpad memory. A log answers "what happened?" A workspace answers "where do things stand right now?" — and for a stateful task, the second question is the only one that matters when deciding the next action. Humans instinctively reach for a workspace: nobody reconciles invoices by re-reading a transcript of every check they've made; they keep a running tally. An agent needs the same affordance, and a plain loop simply doesn't provide one.

⚡ Pro tip: When an agent loses track on long tasks, add a scratchpad before you reach for a bigger model. Working memory is usually the missing piece, not raw capability — the model can reason fine; it just has nowhere to keep its work. A cheap scratchpad often beats an expensive model swap.

The Correct Prompt

The fix was a structured scratchpad the agent maintained as it worked — a live state record separate from the transcript.

Maintain a SCRATCHPAD in this exact format, updating it every turn:

RECONCILED: [item ids already matched and cleared]
FLAGGED: [item id -> reason for each discrepancy]
RUNNING_DISCREPANCY: $[total]
REMAINING: [item ids not yet processed]
NEXT: [the single item you will process this turn]

Read your scratchpad, act on NEXT, then output the updated scratchpad.

What this does: gives the agent an explicit, structured workspace it reads and rewrites each turn, so current state — what's done, what's flagged, the running total, what's left — is always assembled in one place instead of scattered across the transcript.

[object Object], ,[object Object],(,[object Object],):
    scratchpad = init_scratchpad(task)
    ,[object Object], ,[object Object], scratchpad.complete:
        reply = model.call(task, scratchpad=scratchpad.render())
        scratchpad = parse_scratchpad(reply)      ,[object Object],
        persist(scratchpad)                        ,[object Object],
    ,[object Object], scratchpad.result

What this does: threads a structured scratchpad through the loop as the agent's authoritative state, parsing the updated version each turn and persisting it — so the current picture is always explicit, compact, and recoverable rather than reconstructed from dialogue.

Results and What Changed

Success jumped from roughly forty percent to the low eighties, and — unlike the re-reading fix — it held steady as invoices got longer. The scratchpad gave the agent a compact, current picture of its own progress, so it stopped re-checking cleared items and stopped dropping flags. The model didn't get smarter; it got a place to keep its work.

The token cost also dropped, which surprised the team. Re-reading twenty turns of transcript every step had been carrying enormous, growing context; a compact scratchpad that holds only current state is far smaller than the full history it replaces. Better results and lower cost, because a maintained summary of state beats an ever-growing log of events.

This is the counterintuitive economics of working memory: the thing you'd assume adds overhead actually removes it. A scratchpad looks like extra structure the agent has to maintain, so you'd expect it to cost more — but it replaces the far larger cost of re-parsing the whole transcript each turn. You're trading a big, growing input for a small, stable one. On long tasks the scratchpad doesn't just improve accuracy; it caps context growth, which is exactly the runaway cost that makes long agent runs expensive in the first place.

⚡ Pro tip: Cap the scratchpad's size explicitly. If a field like FLAGGED can grow without bound, it reintroduces the context-growth problem you were solving. Set a limit per field and have the agent summarize or archive overflow, so the scratchpad stays a compact working memory rather than slowly becoming another log.

⚡ Pro tip: Make the scratchpad structured, not freeform. A freeform "notes to self" blob drifts and bloats; a fixed schema — named fields the agent must fill each turn — keeps it compact and forces the agent to actually track each category of state instead of narrating vaguely.

How to Apply This to Your Situation

Reach for a scratchpad whenever an agent must track state across many steps: anything with items to process, running totals, accumulating findings, or a plan that evolves. The signal you need one is an agent that loses the thread on long tasks — re-doing work, forgetting decisions, contradicting itself as the run grows.

Design the schema around what the task must not forget. A research agent's scratchpad tracks sources found and questions still open; a coding agent's tracks files changed and tests passing; a planning agent's tracks the plan and which steps are done. The fields are the state that matters, made explicit.

The schema design is where agent scratchpad memory pays or disappoints, so it's worth a few minutes up front. List the specific things your agent forgets on long runs — the re-checked items, the dropped flags, the lost running total — and make each one a required field. If a category of state has caused a bug, it belongs in the schema as a named slot the agent must fill every turn. The schema is essentially a checklist of everything the task can't afford to lose, and writing it down forces you to actually enumerate that list instead of hoping the model tracks it.

⚡ Pro tip: Persist the scratchpad outside the context window, not just inside it. Written to a file or store each turn, it becomes a durable checkpoint — the agent can crash, resume, and pick up exactly where it left off by reloading the scratchpad, because its entire state lives in one recoverable place.

Next Steps

Take an agent that struggles on long, stateful tasks and give it a structured scratchpad: define the fields its task can't afford to lose, have it read and rewrite the scratchpad each turn, and persist it. Run your longest task and watch whether it still loses the thread — it usually stops.

The scratchpad schema and the read-update-persist loop are reusable across every stateful agent. I keep the pattern saved and versioned in PromptABCD, so a new agent gets working memory from the first run — instead of a loop that quietly loses track of its own progress the moment a task runs long enough to matter.