A support bot forgets a customer’s order number three messages after hearing it. A coding assistant loses the file structure you explained ten minutes ago. That forgetfulness is not a personality flaw. It is a hard limit built into the way large language models work.
MemGPT, short for MemoryGPT, is one of the most practical responses to that limit. Introduced in a 2023 paper from UC Berkeley researchers, MemGPT treats an LLM less like a chatbot and more like an operating system. The model gets tools to read and write its own memory, moving information between a small, fast context window and larger external storage. The result: conversations that can last for weeks, documents that can be queried without stuffing every page into the prompt, and agents that remember what happened yesterday.
What MemGPT Actually Does
MemGPT is not a new model. It is a system layer that wraps around an existing LLM, such as GPT-4 or Claude. The wrapper gives the model a memory hierarchy and a set of functions it can call during a conversation.
Those functions include things like core_memory_append, core_memory_replace, archival_memory_insert, archival_memory_search, and conversation_search. When the model wants to remember a fact, it calls a function. When it needs an old detail, it searches for it. The system prompt explains how memory works, and the model decides when to use it.
That decision-making is the key. A standard chatbot passively receives whatever fits in the context window. A MemGPT-style agent actively manages what stays in view and what gets filed away. It can pause mid-response, call a memory tool, read the result, and continue with better information.
The Operating System Analogy, Explained
The analogy is straightforward. A context window is like RAM: fast, expensive, and small. External storage is like a disk: slower, cheaper, and much larger. MemGPT performs virtual context management, paging data between the two.
In practice, the main context holds the system instructions, a small block of core memory, and the most recent messages. Everything else lives outside. Older messages go to recall memory. Documents and notes go to archival memory. When the main context fills up, the system evicts older content. When the model needs that content again, it searches for it.
Core Memory: The Sticky Notes
Core memory is a small, always-visible block of facts. It might include the user’s name, their preferences, the current project, or a persona instruction. Because space is limited, the model has to choose what deserves a sticky note. Example: User is vegetarian. Project deadline is March 15. Prefers concise answers. When a fact changes, the model uses core_memory_replace to update it.
Recall Memory: The Conversation Log
Recall memory stores the full message history outside the context window. It is searchable. If a user asks, ‘What did we decide about the launch date last week?’ the agent can query recall memory and pull the relevant exchange back into view. This is similar to retrieval-augmented generation, but applied to dialogue rather than documents.
Archival Memory: The Filing Cabinet
Archival memory is for external knowledge: PDFs, notes, research papers, meeting transcripts. The agent inserts chunks of text and searches them later. Upload a 200-page contract, and the agent can retrieve the clauses that matter instead of trying to hold the whole document in a prompt.
Why a Fixed Context Window Is Not Enough
Context windows keep growing. Models now offer 128,000 tokens, and some stretch past a million. That sounds like plenty until you use it. A long customer support thread can burn through tens of thousands of tokens. A coding session with file contents and error logs can do the same. A personal assistant that remembers your entire life needs more than a single window.
Bigger windows also cost more. Every token you send increases latency and price. Models can lose track of details buried in the middle of a long prompt. MemGPT addresses those problems by keeping the active context small and relevant. It also lets an agent persist across sessions. The tenth conversation with a user does not have to feel like the first.
MemGPT in Practice: What It Enables
- Long-running companions: A journaling bot that remembers your goals, your mood patterns, and what you said last month.
- Document analysis: A research assistant that reads hundreds of papers and cites the right one when you ask a follow-up question.
- Multi-step agents: A travel planner that tracks flight options, hotel preferences, and budget changes over several days.
- Customer support: A bot that escalates a ticket with full history, including the customer’s previous attempts and promised refunds.
- Personal knowledge management: An assistant that connects notes from meetings, emails, and reading lists without losing the thread.
Building this kind of system takes more than a clever prompt. You need to choose a memory architecture, decide what gets summarized, and test how well retrieval works over time. A practical guide to memory for autonomous LLM agents can help you compare hierarchical memory, vector search, and simple summarization before you commit to one approach.
How MemGPT Compares to RAG and Summarization
Retrieval-augmented generation is great for pulling facts from a knowledge base. It is less suited to maintaining a running conversation state. Summarization compresses old messages into a paragraph, which saves space but loses specifics. MemGPT combines both ideas with explicit memory management. The model decides what to save, what to search, and what to forget.
That flexibility comes with a catch. MemGPT depends on reliable function calling. A model that forgets to save an important detail, or saves too much trivia, will produce a messy memory. Larger models like GPT-4 handle this well. Smaller open-source models often struggle. You also need good schemas, clear instructions, and evaluation harnesses to catch memory failures before users do.
From MemGPT to Letta: The Project Today
The original MemGPT research repository evolved into the open-source Letta framework. Letta packages the same core ideas into a production-ready system with memory blocks, tool use, multi-agent support, and a REST API. If you want to build a stateful agent without reinventing the memory layer, the Letta project is a sensible starting point.
The naming can be confusing. MemGPT is the research concept and the original paper. Letta is the framework and company that grew out of it. Many people still say MemGPT when they mean the broader pattern of LLM-managed memory.
Limits and Open Questions
MemGPT is not a cure-all. It adds complexity, cost, and new failure modes. A few open questions remain:
- Memory accuracy: How do you verify that the agent saved the right fact and not a hallucinated one?
- Privacy: Persistent memory stores personal data. Users need ways to inspect, edit, and delete it.
- Evaluation: What does good memory look like? Accuracy, latency, cost, and user trust all matter.
- Scalability: As archival memory grows, search quality and retrieval speed become critical.
- Model dependence: Strong function-calling models work best, which can lock you into specific providers.
Researchers are exploring hierarchical memory, episodic memory, and memory consolidation. The field is young, and best practices are still forming.
How to Try MemGPT-Style Memory Without a Research Lab
You do not need to reproduce the paper to get the benefits. Start with a simple memory schema: a core block for stable facts, a searchable log for past messages, and a document store for reference material. Give your agent explicit save and search tools. Use a model with solid function-calling support. Set limits on core memory so it does not bloat.
Then test with multi-session scenarios. Ask the agent to recall a detail from three conversations ago. Change a core fact and verify the update. Log every memory operation so you can debug what the model chose to remember. If you want a batteries-included option, look at Letta and adapt its memory blocks to your use case.
The real shift is not just bigger context windows. It is giving models a say in what they remember, what they retrieve, and what they let go. That changes how we design agents, how we evaluate them, and how users experience them. MemGPT is one blueprint for that future, and it is already shaping the tools people build today.