Your agent doesn't need a better memory. It needs to forget.

Every few weeks a new project promises to give your coding agent a real memory. Persistent recall across sessions, a vector database of everything you have ever discussed, compressed observations injected automatically so the agent always remembers your preferences, your past decisions, the shape of your codebase. Some of these tools are genuinely impressive engineering. I installed one of the popular ones, used it for a week, and uninstalled it.

Not because it was broken. Because it was solving the wrong problem, and in solving the wrong problem it made the real one worse.

Storage was never the bottleneck

The pitch for agent memory assumes the failure mode is forgetting: the agent lost the thread because it did not retain enough. So the fix is to retain more. Store every session, embed it, and pull it back in next time.

But think about what actually goes wrong in a long session with an agent. It is almost never that the agent lacks information. It is that the information it needs is drowning in information it does not. The context window is an attention budget, and every token in it competes with every other token. When you auto-inject fifty thousand tokens of compressed history at the start of a session and maybe a fifth of it is relevant to today’s task, you have not helped the agent remember. You have diluted the signal across the entire window and made the relevant part harder to find.

This is the same effect that degrades any over-stuffed context: recall gets worse as the window fills, and information in the middle of a long context is the most likely to be missed. A memory layer that front-loads everything is not fighting that effect. It is feeding it.

Pre-inference versus just-in-time

The cleanest way I have found to think about this is the distinction between two retrieval strategies, which Anthropic lays out well in their post on effective context engineering.

Pre-inference is the memory-plugin model. Before the agent even starts, the system guesses what might be relevant, retrieves it, and injects it. The bet is that the guess is good. When it is wrong, and over a whole session it usually is at least partly wrong, the cost is paid on every subsequent token.

Just-in-time is the opposite. You keep lightweight identifiers in context, file paths, a query, a reference, and let the agent pull the actual content into the window at the moment a task needs it. This is exactly how Claude Code already works without any memory layer bolted on: a stable project file loads up front, and primitives like grep and reading a file let the agent go get what it needs when it needs it. The agent explores its own way to the relevant context instead of having a pile of maybe-relevant context poured on it in advance.

What I actually rely on

Once you stop trying to give the agent a photographic memory, what you need turns out to be modest and mostly already there.

For continuity within a piece of work, I keep a short handoff file: the goal, what is done, what is in progress, what is blocked, and the decisions that would be expensive to relitigate. It is a page, not a database. When I pick the work back up, that page is the memory, and it is high-signal because I wrote it deliberately rather than having it auto-compressed from a transcript.

For long sessions that fill up, compaction does the job that a memory layer claims to do, but at the right moment: when the context is actually getting full, it summarizes and reinitializes while preserving the things that matter, the architectural decisions, the open bugs, the key implementation details. That is retrieval timed to need, not to session start.

And for durable project knowledge, the filesystem is the memory. The code is on disk. The conventions are in a short instructions file. The detail is in reference files the agent opens when the topic comes up. None of it needs to be embedded and re-injected, because the agent can navigate to it.

When a memory layer earns its complexity

I want to be fair to the tools I turned off, because there are real cases where they pay for themselves. Several agents working in parallel that need a shared store of observations. History spanning many repositories and many months at a scale no single handoff file can hold. A team that genuinely needs a common, queryable record of what was tried and why. If you have that problem, the infrastructure is worth it.

But most of us, most of the time, are one person working on one thing, and we reach for a vector database when what we needed was a paragraph of notes and the discipline to write it. Similarity is not relevance. A store that can find the twenty most similar past chunks is not the same as an agent that pulls the one file this step requires. Solve the retrieval design first, with workflow, and add infrastructure only when a real pain shows up that the simple version cannot handle.

The skill an agent needs is not perfect recall. It is the same skill a good engineer has in a long meeting: holding the few things that matter right now and trusting that everything else is a lookup away. Knowing what to leave out is not a weakness of the system. It is the whole point of it.