Persisting Context Across CLI Agent Sessions

Here's a counterintuitive truth about agent memory: the obvious way to give a CLI agent memory across sessions — save the whole conversation and reload it next time — makes the agent worse over time, not better. More saved history means a bloated, expensive, increasingly confused agent that drags last week's dead ends into today's fresh question. Good cli agent session memory is about saving less, not more — the right distilled facts instead of the raw transcript. This teardown takes the naive "save everything" approach and rebuilds it into memory that actually helps.

The distinction sounds small and it changes everything about how the agent behaves on day thirty.

Before: Pickle the Whole Transcript

The version everyone writes first is disarmingly simple. At the end of a session, dump the entire messages list to disk; at startup, load it back.

[object Object], pickle, pathlib

STORE = pathlib.Path.home() / ,[object Object], / ,[object Object],

,[object Object], ,[object Object],(,[object Object],):
    STORE.write_bytes(pickle.dumps(messages))

,[object Object], ,[object Object],():
    ,[object Object], pickle.loads(STORE.read_bytes()) ,[object Object], STORE.exists() ,[object Object], []

What this does: Serializes the full message history — every user turn, every assistant response, every raw tool result — and restores it on the next run. It "works" on day one and quietly rots from there.

Why It Fails

The transcript grows without bound, and every problem flows from that. By the second week, loading history means prepending thousands of tokens to every single request, so each call is slower and costs more — you're paying to resend last Tuesday's directory listing on today's unrelated question.

Then there's staleness. That saved transcript contains facts that were true when captured and aren't anymore: a file that's since been deleted, a test that used to fail and now passes, a decision you later reversed. The model reads all of it as equally current and reasons from stale data with full confidence. An agent that "remembers" a bug you fixed will keep trying to work around it.

And raw tool results are the worst offenders. A single git log or API response can be thousands of tokens of detail the model needed for exactly one turn and never again. Persisting those verbatim is how a memory file balloons to megabytes of noise. Effective cli agent session memory can't be a tape recorder — it has to be a note-taker.

⚠️ Common mistake: Treating "memory" as "the full conversation history." Persisting raw transcripts scales terribly and poisons future sessions with outdated facts. Memory should be a small, curated set of durable conclusions, not an ever-growing log of everything that was ever said.

After: Persist Distilled Memory, Not Raw History

The rebuilt design separates two things the naive version conflated: the ephemeral conversation (which lives and dies within one session) and durable memory (a small set of facts worth carrying forward). Only the second gets persisted, and it gets persisted as structured data, not transcript.

[object Object], json, pathlib
STORE = pathlib.Path.home() / ,[object Object], / ,[object Object],

,[object Object], ,[object Object],():
    ,[object Object], json.loads(STORE.read_text()) ,[object Object], STORE.exists() ,[object Object], {
        ,[object Object],: [], ,[object Object],: [], ,[object Object],: ,[object Object],
    }

,[object Object], ,[object Object],(,[object Object],):
    STORE.parent.mkdir(exist_ok=,[object Object],)
    STORE.write_text(json.dumps(mem, indent=,[object Object],))

What this does: Stores memory as a small JSON object with distinct slots — durable facts, user preferences, and a running summary — instead of a message list. It's human-readable, easy to edit, and stays tiny because it holds conclusions, not conversations.

At session end, you distill what happened into that structure rather than dumping it. A quick model call turns the session into a few durable notes.

[object Object], ,[object Object],(,[object Object],):
    ask = (,[object Object],
           ,[object Object],
           ,[object Object],)
    resp = client.messages.create(
        model=,[object Object],, max_tokens=,[object Object],,
        messages=messages + [{,[object Object],: ,[object Object],, ,[object Object],: ask}],
    )
    new = json.loads(resp.content[,[object Object],].text)
    mem[,[object Object],] = dedupe(mem[,[object Object],] + new[,[object Object],])[:,[object Object],]
    mem[,[object Object],] = dedupe(mem[,[object Object],] + new[,[object Object],])[:,[object Object],]
    ,[object Object], mem

What this does: Asks the model to pull out the handful of lasting facts and preferences from a session, merges them into existing memory, and caps each list so memory can't grow unbounded. Next session starts with these notes, not the whole transcript.

Breaking Down Each Element

The two-store split is the core idea. Conversation is what you're saying right now; memory is what you've learned that outlives this conversation. Keeping them separate means a long, messy debugging session doesn't permanently bloat memory — only its conclusions survive. This mirrors how people work: you don't recall every word of a meeting, you recall the decisions.

The structured slots matter because they let you inject memory precisely. At startup you load memory and fold it into the system prompt as a compact briefing — "known facts about this project: ...; user prefers: ..." — rather than replaying turns. A dozen bullet points of distilled knowledge outperform a thousand lines of transcript, at a fraction of the tokens.

The caps are the guardrail the naive version lacked. Fifty facts, twenty preferences — when memory hits the ceiling, you drop the oldest or, better, ask the model which are still relevant. Bounded memory is the whole point; unbounded memory is the disease you started with.

⚡ Pro tip: Timestamp every fact and let old ones decay. A fact learned three months ago about a fast-moving codebase is a liability, not an asset. Attaching a date lets you age out or re-verify stale memories instead of trusting them forever.

⚡ Pro tip: Make the memory file human-editable and tell users where it lives. When the agent "remembers" something wrong, the fix should be opening ~/.agent/memory.json and deleting a line — not an opaque reset. Plain JSON turns a mysterious behavior into a one-line edit.

What Happens When Memory Contradicts Reality?

There's a failure mode distilled memory introduces that raw transcripts don't, and you have to design for it: a stored fact can become false. Memory says "the auth module lives in auth.py," someone renamed it last week, and now the agent confidently references a file that's gone. Persisted knowledge is a snapshot, and snapshots go stale.

The rule that keeps cli agent session memory trustworthy is simple to state: observation beats memory. When a stored fact conflicts with what a tool actually returns right now, the fresh observation wins, and the stale fact gets corrected — not the other way around.

[object Object], ,[object Object],(,[object Object],):
    ,[object Object],
    ,[object Object], fact ,[object Object], ,[object Object],(mem[,[object Object],]):
        ,[object Object], contradicts(fact, observation):
            mem[,[object Object],].remove(fact)
            mem[,[object Object],].append(,[object Object],)
    ,[object Object], mem

What this does: Checks new tool observations against stored facts and replaces any that are contradicted by current reality. Memory self-heals as the agent works, so a rename or a fix updates the record instead of poisoning future sessions with an outdated claim.

The framing to give the model matters too. Inject memory as "here's what I believed at the start of this session, verify before relying on it" rather than as ground truth. A model told its memory is provisional will double-check a stale fact against reality; a model told its memory is fact will build on the error. Treat persisted knowledge as a strong prior, not gospel.

⚡ Pro tip: Log every memory correction. When facts keep getting corrected in the same area, that's a signal the codebase is churning there — useful information in its own right, and a hint that those facts maybe shouldn't be persisted at all until things stabilize.

Scoping Memory to the Right Context

Global memory is often the wrong grain. An agent used across five projects shouldn't apply project A's facts to project B. The fix is to key memory by context — usually the working directory — so each project gets its own notes.

[object Object], ,[object Object],(,[object Object],):
    ,[object Object], hashlib
    key = hashlib.sha1(,[object Object],(cwd).encode()).hexdigest()[:,[object Object],]
    ,[object Object], pathlib.Path.home() / ,[object Object], / ,[object Object], / ,[object Object],

What this does: Derives a per-directory memory file from the current path, so the agent loads the memory relevant to where you're working rather than a single global blob. Project-scoped memory keeps facts from bleeding across unrelated codebases.

This scoping is what makes memory feel intelligent instead of intrusive. Run the agent in your API repo and it recalls that repo's quirks; run it in your data pipeline and it recalls that one's — no cross-contamination.

⚡ Pro tip: Add a project-level shared memory file the team can commit to version control, separate from personal memory. Shared facts about the codebase live in the repo; personal preferences stay local. New teammates then inherit the agent's accumulated project knowledge on their first run.

Variations for Different Contexts

A backend developer persists architectural decisions — "we use repository pattern, not active record" — so the agent stops re-suggesting the style the team rejected months ago.

A data scientist stores dataset quirks as memory: which columns are unreliable, which join keys are safe, so every new analysis session starts already aware of the traps that burned the last one.

A support engineer keeps a memory of resolved-issue patterns, so recurring customer problems get matched against past resolutions instead of re-diagnosed from scratch each time.

A DevOps engineer persists the shape of their infrastructure — which services are noisy, which alerts are usually false alarms — so an incident agent starts each session already calibrated to the environment's quirks instead of treating every page as novel.

Each stores different content, but the shape is identical: distill sessions into durable facts, scope them to context, cap their growth, and inject them as a briefing rather than a replay.

Save and Reuse This

The memory schema and the distillation prompt are the real assets here — the exact instructions that reliably pull durable facts out of a session without hoarding noise took real tuning to get right. Rewriting that extraction prompt from memory each time you build a new agent means relearning what "durable" means all over again.

Keeping the distillation prompt and the memory structure in a library like PromptABCD means your next agent inherits a memory system that already knows how to forget the right things. The pickle-the-transcript version you can write in your sleep; the prompt that turns a messy session into three clean, lasting facts is the piece worth keeping forever.