Three vendors, three polished demos, three claims that their agent platform is “production ready.” A team I spoke with last spring ran all three side by side for six weeks before signing anything. The winner wasn’t the one with the slickest demo video. It was the one whose audit logs got their security reviewer to stop asking questions by week two.
That is the shape of most honest AI agent platform comparisons. Every product page promises autonomous reasoning. The real differences show up in tool permissions, p95 latency, token accounting, and what happens when a third-party API returns a 429 halfway through a workflow.
Compare the task shape before you compare the platforms
An agent that answers questions from a knowledge base and an agent that issues refunds across four internal systems are not the same problem, and no single platform wins both. Before you open a single comparison spreadsheet, write down what your agent actually has to do.
In the deployments I’ve tracked, most useful agents fall into a range of 5 to 15 tool calls per completed task. A support triage agent might need three: look up the customer, check order status, draft a reply. A procurement agent might need twelve, spanning an ERP, a vendor portal, an email thread, and a policy database. That gap matters more than any feature matrix, because the cost and failure rate of a multi-step run compound with every additional hop.
Write the number down. Five tool calls and twelve tool calls lead to different shortlists.
The main contenders and where they genuinely diverge
Broadly, the market splits into three camps, and the split is about control versus convenience rather than quality.
Code-first orchestration: LangGraph, CrewAI, AutoGen
LangGraph models an agent as an explicit graph of nodes and edges with persistent state and checkpointing, so you can pause a run, inspect it, and resume. That structure is why teams with compliance requirements gravitate toward it. CrewAI gets you to a working multi-agent prototype in an afternoon using role-based crews, which is a genuine advantage for exploration and a liability if the abstraction hides the failure you need to debug. AutoGen, Microsoft’s research-flavoured framework, leans conversational and multi-agent; it has been rewritten more than once, so pin your version.
All three expect you to write Python. If nobody on your team does, stop reading this section.
Vendor-native platforms: Copilot Studio, Agentforce, Vertex AI Agent Builder, Bedrock Agents
Microsoft Copilot Studio wins where the agent lives inside Teams and Microsoft 365, largely because identity, connectors, and governance come from Entra rather than from you building them. Salesforce Agentforce is compelling if your agent needs CRM context, and close to pointless if it doesn’t. Google’s Vertex AI Agent Builder and ADK suit teams already on GCP who want grounding and a managed runtime. Amazon Bedrock Agents is the IAM-native option, which is either a blessing or a wall depending on your ops team.
The trade is real: you get identity, logging, and deployment for free, and you get a ceiling on how weird your orchestration can get.
Glue-layer tools: n8n, Zapier, Make
These aren’t agent platforms in the LLM sense, but they handle a surprising number of real jobs, especially where the agent is mostly deterministic with one or two model calls. Cheap, observable, and easy for non-engineers to maintain.
Pricing models are where comparison spreadsheets fall apart
Four billing structures dominate, and they produce wildly different numbers for the same workload:
- Per seat. Predictable, and quietly punishing once you have 40 support reps triggering the same agent.
- Per token, passed through. Transparent, but a reasoning-heavy agent with long context can cost 30x a lean one on the same task.
- Per execution or per flow run. Fine until a retry loop fires six times on one failed step.
- Per conversation. Attractive for chat, awkward for anything that isn’t chat, and sensitive to how talkative your agent is.
Take Salesforce Agentforce’s list pricing around $2 per conversation. If your agent resolves in four messages, that’s reasonable. If your agent asks clarifying questions seven times before resolving, you’re paying twice for the same outcome. Run your own average message count per resolution before you model anything.
The tool layer decides whether the comparison even matters
Here’s the part that rarely makes it into analyst reports: most production failures aren’t reasoning failures. They’re tool failures. A malformed JSON payload, a missing rate limit, a write action that fired twice.
The layer of tools and permissions around your agent is what determines whether a bad step becomes a bad day. During evaluation, ask each vendor three questions: can a tool be marked read-only and enforced at the platform level, can actions require human approval before execution, and can you replay a failed run against a sandbox with the same inputs? Products that answer all three are in a different category from products that answer none.
Load testing is the step almost everyone skips
Demos run at one request per minute. Your queue runs at forty. Latency behaves very differently at that scale, particularly when each task fans out into a dozen sequential model and tool calls.
Ask for p95, not average. A platform averaging 3 seconds with a p95 of 22 seconds will feel broken to users even though the mean looks fine. For a deeper look at how the enterprise options behave under sustained production traffic, this breakdown of which enterprise AI agents hold up under real production load is worth reading before you shortlist.
Which platform fits which team
- Two engineers, six weeks, internal tool: LangGraph or CrewAI, deployed on your own infra. Skip the enterprise contracts.
- Regulated industry with a security review: Copilot Studio or Bedrock Agents, because the identity and audit story is already built.
- Sales or service team living in a CRM: Agentforce, if the CRM is Salesforce and the data is clean.
- Ops team automating back-office chores: n8n with two or three model calls bolted in. Far cheaper than you expect.
- Anything customer-facing at volume: whatever you pick, budget for an evals harness before you budget for the licence.
A two-week bake-off that beats any comparison article
Pick twenty real tasks from your logs, including the ugly ones. Run them through two platforms. Log success rate, cost per completed task, median and p95 latency, and how often a human had to step in. Twenty tasks is enough to separate a platform that works from one that demos well. The approach in this guide to picking AI agents without trusting the demo video maps neatly onto platform selection too.
One warning from watching these bake-offs: the platform that wins on prototype speed usually loses on debugging time. Budget the second week purely for fixing what broke.
The clause worth negotiating before you sign
Every serious contract conversation should end with a data portability question: if we leave in eighteen months, do we get our prompts, tool definitions, eval sets, and run history out in a usable format? Agent platforms accumulate institutional knowledge in exactly those artefacts, and vendors know it. Some will export. Some will export a JSON blob that only their runtime can read.
Ask for a sample export before the pilot, not after. Teams that did this reported switching costs measured in days. Teams that didn’t described rebuilding months of tuning from scratch, and most of them simply stayed put and paid the renewal. That single question tells you more about a platform’s long-term economics than any feature grid, and it costs you one email to ask.