Ask a large language model to remember your birthday and it will happily agree. Ask it again next week and you are a stranger. That gap between what a model understands and what it retains is the problem Letta was built to solve.
Letta is an open-source framework for building AI agents that keep state over time. Instead of stuffing a conversation into a prompt and hoping the important parts survive, you give an agent memory it can read, edit, and search, then run it as a persistent service that picks up where it left off. It’s a different way of thinking about agent architecture, and it’s one of the more interesting ideas to come out of the agent-tooling boom.
What Letta Actually Is
The project started as MemGPT, a research effort at UC Berkeley’s Sky Computing Lab. The 2023 paper framed an unusual analogy: treat the LLM like a CPU, the context window like RAM, and external storage like a disk. Just as an operating system pages data in and out of memory, an agent should manage what it holds in context and what it parks elsewhere.
That research became a company, Letta Inc., founded in 2024 by Charles Packer and Sarah Wooders with seed funding led by Felicis. What they shipped is three things bundled together:
- A server that runs agents as long-lived processes behind a REST API.
- SDKs for Python and TypeScript so you can create, message, and inspect agents from your own code.
- An agent specification describing which model to use, which tools the agent can call, and how its memory is structured.
State lives in a database, SQLite by default and Postgres for anything serious. Your application doesn’t hold the conversation history; the agent does.
Memory Blocks: The Core Idea
Letta’s memory model is refreshingly plain. A memory block is a labelled chunk of text with a character limit, and it sits in the context window on every call.
Core memory
Blocks named something like persona and human hold the essentials: how the agent should behave, who it’s talking to, ongoing projects, preferences. The agent can rewrite them mid-conversation using tools such as core_memory_append and core_memory_replace. That’s the trick. The model isn’t passively fed a summary someone else wrote; it decides what’s worth keeping.
Archival and recall memory
Anything too bulky for core memory goes to archival storage, which the agent queries with a search tool when it needs a detail. A separate recall layer lets it search back through past messages. Together they give an agent three tiers of memory working at different speeds, much like human recall.
Sleep-time agents
A later addition worth knowing about: a background agent that reorganises and consolidates memory while the main agent is idle. Rather than pausing a live conversation to rewrite blocks, the work happens off to the side. It reduces latency during chats and keeps long-running memory tidier.
Why not just use a bigger context window
Models now handle hundreds of thousands of tokens, so why bother? Three reasons. Cost scales with context, so carrying 80,000 tokens of history on every request gets expensive fast. Retrieval degrades in long contexts, with details buried in the middle getting missed more often than anyone would like. And a persistent, inspectable memory is auditable in a way a sprawling transcript isn’t. You can open a block and see exactly what the agent believes about you.
How It Differs From Stateless Agent Frameworks
Most popular agent libraries treat a run as a discrete event. You build a chain or a graph, execute it, get a result, and everything is discarded or dumped into a log.
- Lifecycle: LangGraph and similar tools orchestrate a workflow. Letta hosts a persistent entity with an identity that survives restarts.
- Memory model: elsewhere you bolt on a vector store or a summariser. Letta makes memory a first-class primitive the model itself edits.
- Deployment shape: frameworks live inside your app. Letta runs alongside it as a service other clients can reach.
- Model choice: Letta is model-agnostic, working with OpenAI, Anthropic, Google Gemini, Groq, vLLM, Ollama, and others through a single interface.
That last point matters more than it sounds. Swapping a model is a config change, not a rewrite, which makes it easy to test whether a cheaper model handles your workload.
The Agent File and the ADE
Two features deserve separate mention because they change how you work day to day.
The Agent File (.af) is a serialised agent: model settings, tools, and the contents of every memory block in one portable file. You can commit it to a repo, version it, hand it to a colleague, or restore a known-good configuration after an experiment goes sideways. Treating an agent as an artifact rather than a running process is a genuinely useful shift.
The Agent Development Environment is a browser interface that shows memory blocks updating live as you talk to an agent. Watching a block get rewritten in real time is the fastest way to understand why an agent behaves the way it does. When something goes wrong, the answer is usually visible in the memory.
Running Letta on Your Own Machine
Getting started is quick. There’s a hosted cloud option, but local is where most people experiment first. A single Docker command pulls the server image and exposes it on port 8283; a pip install gives you a command-line interface with a letta run entry point that spins up an agent in seconds.
The model-agnostic design means you can point it at a small open-weight model running locally through Ollama or vLLM. For a personal agent that handles reminders, notes, and preferences, that’s often plenty, and the push toward compact, task-focused models keeps widening the range of what fits on a laptop. Work like PrismML’s effort to make tiny LLMs genuinely useful points in the same direction: agents that run cheaply and privately rather than renting a frontier model for every trivial request.
What People Build With It
The obvious use case is a personal assistant that remembers your projects, your preferences, and the context of conversations from months ago. Beyond that, some patterns show up repeatedly.
Support agents that retain account history across sessions, so a customer never repeats themselves. Research agents that accumulate findings in archival memory over days and only pull the relevant pieces into context. Tutors that track what a learner has already mastered. Multi-agent setups where several agents share memory blocks, using them as a common blackboard so a planner and an executor stay in sync.
Each of these leans on the same property: the agent persists, and its knowledge compounds.
Rough Edges Worth Knowing About
Self-editing memory is powerful and also unpredictable. An agent can write a wrong assumption into a core block and act on it confidently for weeks. Memory needs the same care as any other prompt surface: clear block descriptions, sensible character limits, and regular inspection.
Large blocks eat tokens on every call, so an agent with five verbose memory blocks is quietly more expensive than it looks. The API has moved fast, with breaking changes between releases that will bite anyone pinning loosely. And evaluating a stateful agent is genuinely harder than testing a stateless one, because behaviour depends on accumulated history rather than a single input. You need conversations that span sessions, not one-shot test cases.
A Sensible First Project
Start with something small and personal rather than a customer-facing system. A note-taking agent that remembers your ongoing projects is a good fit, because you can judge its memory quality yourself within a day.
Give it two core memory blocks and reasonable character limits. Talk to it for a week in the ADE so you can watch the blocks change. Note where it records something useless, where it forgets something obvious, and how often archival search actually fires. Then export the Agent File before you change anything, so you have a checkpoint to return to.
Once a personal agent holds up over a couple of weeks of real use, you’ll have a much better sense of whether your production use case needs persisted memory at all. Plenty don’t. For the ones that do, having a framework that treats memory as a design decision instead of an afterthought saves a lot of awkward patching later.