Memory is the persistent store that lets an agent pick up where it left off across runs, sessions, and reboots.
Memory is what turns a single prompt into a long-running relationship. Without memory, the agent forgets every conversation the moment the session ends. With memory, the agent remembers the user's preferences, the previous decisions, the open loops, and the things to avoid.
The three layers of memory
Agent memory typically has three layers, each with a different durability and access pattern:
1. Working memory. The current context window. Fastest to access, smallest in capacity, resets when the session ends. The working memory is what the model sees in the current inference call.
2. Session memory. The conversation log for the current session. Faster than long-term storage, larger than the working memory, but still scoped to a single session. The session memory is what survives across multiple agent loops within the same session.
3. Long-term memory. The persistent store that survives across sessions, reboots, and process restarts. Slower to access, much larger in capacity, and the key to the agent's long-running relationship.
Why memory is hard
Memory is one of the hardest parts of agent design because memory is where the cost, the latency, and the relevance trade-offs collide. The wrong memory (too much, too little, too noisy, too stale) degrades the agent's performance more than any other single component. The right memory is the difference between an agent that feels stateful and an agent that feels like a goldfish.
Operator implications
Memory is the right place to start when an operator is debugging an agent that is forgetting things it should remember, or remembering things it should forget. The most common operator issues with memory are: the memory is too lossy (the agent forgets important context), the memory is too noisy (the agent retrieves irrelevant context), the memory is too slow (the agent times out before the memory is loaded), and the memory is too stale (the agent uses outdated information).
Related terms
Memory is the persistence layer that the agent loop writes to and reads from. Memory is the input to the context vs memory trade-off. Memory is the layer that compaction manages. Memory is often implemented as embedding search or as a graph-based memory system.
For the full primer, see Memory & State.
Why memory is the hardest part
Memory is the most failure-prone part of agent design because memory is the only component that survives across iterations, across sessions, and across reboots. Everything else (the working memory, the algorithm, the tools) can be reset; the long-term memory cannot. The long-term memory is also the only component that an external operator can audit: the operator can read the memory, the operator can edit the memory, and the operator can decide what to add or remove.
The operator's interface to the memory is one of the most important product decisions in agent design. The wrong interface (the agent is the only one that can read and write the memory) makes the memory opaque; the right interface (the operator can inspect, edit, and explain the memory) makes the memory manageable. The right interface is the difference between an agent that the operator trusts and an agent that the operator does not.