Open Source AI Agents: What You Get, What It Costs, and Where They Break

A colleague of mine replaced a $2,400-a-month SaaS seat with an agent running on a spare Mac mini. It handles roughly 300 support emails a week, and nobody has asked for the old tool back. It also broke twice in the first month, once by replying to the same customer eleven times in four minutes. Both things are true, and that is roughly the shape of open source AI agents right now.

The appeal is easy to state: no per-seat pricing, no vendor deciding which integrations you’re allowed to touch, no risk of your workflow vanishing in a product pivot. The catch is that “open source” describes the licence on the code, not the effort required to run it.

What open source actually covers

An agent is rarely one piece of software. It’s a model, a loop that decides what to do next, a set of tools (search, code execution, your CRM), some memory, and a way to see what happened afterwards. Open source almost never covers all five layers at once.

The model layer matters most. Open source in model-land usually means open weights: you can download Llama, Qwen, Mistral or DeepSeek checkpoints and run them, but the training data and process stay private. That’s workable for most commercial use, though Llama’s community licence has user thresholds and naming rules worth reading before you ship anything.

Licences on the code are usually simpler. MIT and Apache 2.0 do what you’d expect. The ones to watch are source-available projects, where a handful of popular workflow tools restrict you from hosting them as a competing service. If your plan involves reselling, five minutes with the licence text beats a year with a lawyer.

The frameworks worth your evening

General orchestration

Three names come up constantly, and the differences between them are real.

• LangGraph models an agent as a graph of nodes with explicit state. More setup than most people expect, but the human-in-the-loop checkpoints and the ability to resume a run from saved state are what let it survive production.
• CrewAI is the fastest route to a working demo. You describe roles, hand them tools, and a crew passes work between them. Demo to dependable is where the real work begins.
• AutoGen, out of Microsoft Research, leans into conversation between agents. Excellent for reasoning-heavy research tasks, less so for a tight order-processing pipeline.

Frameworks built for one hard problem

Some of the most useful projects skip general orchestration entirely and solve a single thing that everyone building agents eventually needs.

Letta gives an agent persistent memory backed by a real database, with memory blocks the agent can edit itself rather than a bolted-on vector store. Pipecat handles real-time voice, including the unglamorous parts like voice activity detection and interruption, where a 600ms delay is the difference between natural and unusable. IBM’s BeeAI toolkit covers packaging and deployment, plus agent-to-agent communication. OpenHands, MIT-licensed, is built for coding tasks and sits among the strongest open agents on SWE-bench Verified.

Where the money actually goes

Open source agents don’t delete cost. They move it, and the line items are predictable once you know to look for them.

• Inference. A mid-sized agent loops eight to fifteen times per task. At GPT-4o-class rates, a task costing 40 cents is normal. Ten thousand tasks a month lands you in four figures, which is often the bill you were trying to escape.
• Engineering. The demo takes a weekend. Going from 70% to 95% reliability takes six to ten weeks from someone who knows what they’re doing.
• Observability. You need traces. Langfuse self-hosts for free, and wiring it up takes a day. Keeping it useful is ongoing.
• Evaluation. Fifty real tasks with expected outcomes, run on every change. Teams skip this and then can’t explain why version 14 is worse than version 9.
• On-call. Agents fail at odd hours, usually silently. Somebody has to own the pager.

Small models, tight scopes

The biggest single lever on both cost and reliability is scope. A booking agent that only reads a calendar and writes a reservation will run happily on a 7B model quantised to 4 bits on a consumer GPU, and it will be faster and cheaper than a frontier model handed the same job with a vague prompt.

Local inference has a hard ceiling, though. A 70B model at 4-bit wants about 40GB of VRAM. Below that, multi-step reasoning quality drops off fast and agents start looping: calling the same tool, getting the same error, trying again. If your task needs more than three sequential tool calls, budget for API calls or a rented A100 instead of bullying a small model into compliance.

Memory, or why your agent has amnesia

Most agents built this week have no memory at all. Every conversation starts from zero, the context window gets refilled with the same documents, and the agent cheerfully re-asks a question the user answered yesterday. That’s usually a design choice rather than a bug, and it’s often the wrong one.

Persistent memory is its own engineering problem: deciding what to store, when to summarise, and how to retrieve the right fragment without dragging 40,000 tokens of noise into the prompt. The failure modes are well understood at this point, and the reasons AI agents forget are worth reading before you design a schema you’ll regret. Memory done properly is often the difference between a tool people use daily and one they abandon after a fortnight.

When paying for a closed agent is the smarter call

There’s a point where the honest answer is to buy. If your workflow is common (inbox triage, meeting notes, first-line support) and your volume is modest, a closed product will cost less than the engineering hours you’d sink into an open alternative. The maths flips when you have unusual data, hard compliance boundaries, or enough volume that per-seat pricing stops making sense. Knowing which paid agents earn their subscription saves a lot of wasted evaluation time either way.

A starting stack that survives contact with users

If you’re going to build, start boring.

1. Pick one workflow with a measurable success criterion. “Reply correctly to refund requests” beats “be a helpful assistant” every time.
2. Assemble 50 real tasks with expected outcomes before you write a line of agent code.
3. Use LangGraph or CrewAI with the smallest model that clears 85% on your eval set, then step up only where it fails.
4. Add memory once the stateless version works, not before.
5. Put traces in front of you from day one, not after the first incident.
6. Run it in shadow mode for two weeks: the agent drafts, a human sends, you compare.

That last step catches more disasters than anything else on this list. When the drafts match what your team would have written 90% of the time across two weeks, you have something worth automating. Until then, you have a demo, and demos are cheap.