Agent memory
What an agent keeps in view, what it files away, what it looks up later, and what it does when the facts change.
Read first:The agent loopEmbeddings
Step 1 of 6· Illustrative conversation, facts and scores; MemGPT's thresholds and the Generative Agents retrieval formula are real
A language model remembers nothing between calls. Everything it knows about you in a conversation sits in its context window, a fixed-size working memory that it rereads every turn and that eventually fills. Agents add long-term memory outside the model: a store of facts pulled from past conversations, a running summary of older messages, or a graph linking people, places and things. After each exchange the system decides what is worth writing down; before each answer it fetches the few memories that look most useful and puts them back in view. The hard part is change: a new fact that contradicts an old one has to update or retire it.
Why it matters for your product
Memory decides whether an assistant feels attentive or forgetful, and it drives cost. Resending a whole history every turn is slow: Mem0's authors report over 90% lower token cost and 91% lower p95 latency than full-context prompting, for a small loss in accuracy. Memory also carries risk. Stored personal data needs a retention policy and a way to delete it, and a wrong memory resurfaces in every later session. Test memory with facts that change, not just recall: contradictions are where memory systems quietly go wrong.
For engineersShow the maths
recency = 0.995ʰ
h is the number of hours since the memory was last retrieved. Each unused hour multiplies its recency by 0.995, so memories fade unless they are used; using one resets its clock.
Worked example: After a day: 0.995²⁴ ≈ 0.89. After a week: 0.995¹⁶⁸ ≈ 0.43.
score = α₁·recency + α₂·importance + α₃·relevance
Each part is first scaled to 0–1 across the candidate memories. Importance is the model's own 1–10 rating when the memory is written; relevance is the cosine similarity between the memory's embedding and the question's. Generative Agents set every α to 1 and kept the top memories that fit in the context.
Worked example: Scaled parts of 0.99, 0.63 and 0.86 give 2.48, the top score for the cooking question in the demo.
warn at 0.7 × window; at 1.0 × window, evict ≈ 0.5 × window
MemGPT's example eviction policy: a memory-pressure warning gives the model a chance to save key facts, then a flush moves the oldest messages out and folds them into the running summary.
Worked example: With an 8,000-token window: a warning at 5,600 tokens, and at 8,000 about 4,000 tokens of the oldest messages leave the context.
Memory is only as good as what was written and what retrieval ranks highly: a missed fact, a wrong extraction or a stale memory is repeated confidently in later sessions. Bigger windows are not a full fix either, as models use information in the middle of a long context less reliably than at its start or end.
The words you will hear
Context window
Primary sources
- 01Cognitive Architectures for Language AgentsSumers et al. · 2023
- 02MemGPT: Towards LLMs as Operating SystemsPacker et al. · 2023
- 03Generative Agents: Interactive Simulacra of Human BehaviorPark et al. · 2023
- 04Mem0: Building Production-Ready AI Agents with Scalable Long-Term MemoryChhikara et al. · 2025
- 05Zep: A Temporal Knowledge Graph Architecture for Agent MemoryRasmussen et al. · 2025
- 06Lost in the Middle: How Language Models Use Long ContextsLiu et al. · 2023
Connected ideas
Retrieval and search
Retrieval-augmented generation (RAG)
How a model answers from your documents, and shows you exactly where each claim came from.
ExploreRetrieval and search
Hybrid search and reranking
Search by exact words and by meaning at once, merge the two lists, then let a slower model reread the best few.
ExploreInterpretability and safety
Prompt injection
When text the model reads on your behalf starts giving it orders.
ExploreObservability
Evals
How to know a change made your AI system better, before your users find out it did not.
ExploreWant this working on your data?
We design and build the systems these ideas power: retrieval, agents, voice and the models behind them. Start with a free discovery call.