A support agent receives a message: “Order 4471 still hasn’t arrived.” Less than a second later it does something a plain chatbot cannot. It emits a structured request to look up the order, reads back a shipping record stuck at a Memphis hub, checks the carrier’s exception code, and files a reship. Nobody typed a command. The model chose the action and formatted the arguments itself.
That handoff, from language to an executable call, is what people mean by AI agent function calling. It’s the mechanism that turns a chat model into something that can touch the real world. It’s also where most production agents quietly fall over.
Function calling is a contract, not a capability
The model never runs your code. It can’t reach into your database or hit your CRM. What it can do is read a list of function descriptions you supply and respond with a name plus a JSON payload along the lines of {“order_id”: “4471”}. Your application then decides whether that call is legitimate, executes it or refuses, and hands the result back.
Think of it as a very literal translator. You describe the tools, the model picks one and fills in the blanks, and your runtime plays the adult in the room: performing the work and checking the consequences.
The round trip, step by step
- You define the tools. Each one gets a name, a description, and a JSON Schema for its parameters, including which fields are required.
- The user’s message goes up with those definitions. The model sees the full menu alongside the request.
- The model returns a tool call. Zero or more of them, each with a name and arguments. Sometimes it just answers in prose instead, and that’s a perfectly valid outcome.
- Your runtime validates and executes. Schema check, permission check, business rules, then the actual function.
- The result goes back as a tool message. Success or failure, written in a compact, readable shape the model can reason over.
- The model continues. It might answer the user, call another function, or stop and ask for clarification.
Every agent framework is doing some variation of this loop, whether it’s a sixty-line script or a hosted orchestration service. The mechanics of the handoff, and why it’s the line between a chatbot and a system that does real work, are covered well in this piece on AI agent tool calling.
The schema does more work than the prompt
Teams spend three days polishing a system prompt and ten minutes on tool definitions. That ratio is backwards.
Descriptions are documentation written for a model
“Gets order info” tells the model almost nothing. “Returns status, carrier, and last scan event for a single order. Use when a customer asks where a package is. Requires a four-digit numeric order ID” gives it something to match against. Vague descriptions produce vague tool choice, and you’ll blame the model for it.
Types are guardrails
If quantity is an integer from 1 to 10, set a minimum and maximum. If a field must be one of three strings, use an enum. Every constraint you leave out of the schema becomes either a validation error you patch later or a bad write to a production table.
Fewer tools beat a bigger menu
An agent with eight sharp, well-separated tools picks the right one far more often than an agent with forty overlapping ones. If two tools sound alike to a human reading the descriptions, they’ll confuse the model too. Merge them, or hide the rarely used ones behind a search step.
Function calling, tool calling, and other names for the same thing
The vocabulary shifts depending on who’s talking. Function calling usually means model-generated structured invocations of your own code. Tool calling is the broader umbrella and includes things the provider runs for you, such as web search, retrieval, or a code interpreter. Underneath all of it sits the tool layer itself, which handles registration, permissions, and execution, and that layer is what determines whether your agent works or wrecks something.
Where it breaks in production
Fabricated arguments
A model will invent an identifier that looks plausible. The customer says 4471 and the call arrives with 4472 because the digits read better that way in context. Validate every model-generated ID against a real record before it reaches a write path.
Silent failures
Your function returns HTTP 200 with a body like ok: false, reason: card declined. The agent sees a successful response and cheerfully tells the customer the refund went through. Return errors as explicit, plain-language strings the model can actually act on.
Loops
The same failing call, six times in a row, each with identical arguments. Cap iterations at something generous like five, and change the signal the model sees on the second attempt so it stops repeating itself. This is the single most common reason an agent never finishes a task.
Stale state
An agent fetches a balance on turn one and reasons about it on turn nine, after two writes have changed it. Anything that moves needs re-fetching, not remembering. The gap between an instruction and an action is where these assumptions hide, which is why it pays to understand AI agent reasoning before you trust a long conversation.
Guardrails worth building before you ship
- Read-only by default. Write tools require explicit opt-in per session.
- Idempotency keys on every side-effecting call, so a retry doesn’t send the same email twice.
- Human approval for anything irreversible: refunds above a threshold, record deletion, outbound email to a real customer.
- A dry-run mode that logs the call instead of executing it, useful for evaluating new tools against old traffic.
- Hard timeouts plus a maximum tool-call budget per conversation.
If you can’t describe the worst outcome of a tool firing with wrong arguments, that tool isn’t ready. A refund of 40 dollars and a refund of 40,000 dollars are the same call shape with a different number, and only your runtime knows the difference.
Testing tool selection like it’s a real feature
Tool selection accuracy is measurable, and almost nobody measures it. Build a set of fifty realistic user messages with the tool and arguments you expect for each. Run that suite after every prompt or schema tweak. You’ll catch the regressions that appear when someone adds a ninth tool and quietly drops selection accuracy from 94 percent to 81.
Mock the underlying functions so tests don’t hit real APIs, record full traces from staging so you can replay failures, and include adversarial arguments: negative quantities, dates in the past, strings where integers belong. If you’re building the loop yourself, the patterns in building AI agents with Python cover most of the plumbing without hiding it behind a framework.
Start with the boring functions
The two or three most boring functions you own are the right place to begin: order lookup, calendar availability, ticket status. All read-only, all easy to verify, all with obvious right answers. Get those to near-perfect reliability, watch the traces for a week, then introduce one write.
Agents that ship are rarely the ones with the most impressive tool catalogs. They’re the ones where each function has a narrow job, a description a stranger could follow, and a runtime that refuses to execute anything it can’t justify. Function calling gives a model hands. What it does with them is entirely a design decision, and that decision is yours, not the model’s.