You switch on the Zendesk AI agent on a Tuesday. By Friday it has told three customers your return window is 90 days, promised a same-day replacement your warehouse can’t ship, and signed off with “Best regards, Sarah.” There is no Sarah on your team.
The bot isn’t broken. The setup was. Most teams treat the AI agent as a switch to flip after a slick demo, when it behaves far more like a new support hire: it reads everything you give it in seconds, never asks a clarifying question, and confidently fills any gap you left open. Here’s how to configure one properly, in the order that actually matters.
Start With the Ten Tickets You’d Hand a New Hire
Resist the urge to point the agent at your entire queue. Pull the last 30 days of tickets and filter for three things: high volume, low variation, and answers that don’t change depending on who is asking. On a typical ecommerce or SaaS help desk, that shortlist looks like this:
- “Where is my order?” and “Has my parcel shipped?”
- Password resets and two-factor lockouts
- Updating a billing address or payment method
- Plan upgrades and downgrades
- How to export data or locate a specific report
Those five intents can be 30 to 40 percent of a mid-size queue. A useful filter: if a competent new hire could handle it in their first week with one page of notes, the AI agent can too. Anything needing judgment, negotiation, or an account-level exception should stay human for now. If you want the plumbing behind this, our breakdown of how the Zendesk AI agent works covers how intents, retrieval, and replies are assembled.
Rewrite Your Answers for a Machine That Reads Once
Most help centres are written for humans who skim and search. An AI agent retrieves a chunk of content, reads it a single time, and answers from it. Long, hedge-filled articles are the biggest single cause of vague bot replies.
Split them. One question per article, the answer in the first sentence, the details after. Watch the difference:
Before: “Our returns policy has evolved over the years to better serve our customers. Generally speaking, most items may be returned within a certain window provided they are in resalable condition…”
After: “You can return unused items within 30 days of delivery for a full refund. Start a return from Account > Orders > Return item. Refunds take 5 to 7 business days after the item reaches our warehouse.”
The second version hands the agent a fact it can quote without guessing. It also matters more than most teams realise that published content shapes what a model repeats. We’ve written about how AI answers can be influenced through structure and phrasing, and the same mechanics apply inside your own knowledge base.
Write the Agent’s Instructions Like an Onboarding Doc
The instruction field is not the place for a sentence or two. Treat it as page one of a new hire’s handbook: tone, scope, hard limits, and what to do when stuck.
A worked example for a bike retailer
Something like this beats “be helpful and friendly” by a wide margin:
"You are Northline Bikes’ support assistant. Answer only questions about orders, shipping, returns, and product sizing. Use a warm, plain tone and keep replies under 120 words. Always confirm the order number before discussing order status. Never quote delivery dates beyond what the tracking link shows. Never offer discounts, replacements, or refunds above $150; hand those to a human. If the customer mentions a damaged frame, a safety concern, or a chargeback, escalate immediately without answering."
Four rules, four boundaries, one escalation trigger. Short instructions with explicit “never” lines outperform long, poetic ones because there is less room left to improvise.
Decide the Handoff Rules Before You Touch Automation
Escalation design is where AI support projects live or die. The agent should hand over on signals, not vibes. Build triggers around:
- A sentiment drop, or a second “no, that’s not what I asked”
- Two failed resolution attempts on the same ticket
- High-stakes words: chargeback, lawyer, cancellation, safety
- Order or account value above a threshold you set
The handoff has to carry context. A customer who must retype their problem to a human is already annoyed. Configure the transfer to push a summary, the steps the agent tried, and the relevant order details into the ticket, then route on skill. Teams that do this well often pair it with predictive triage tools like Forethought AI, which sort incoming work before an agent ever opens it.
Replay 50 Real Tickets Before You Go Live
Don’t test with five scenarios you invented. Export 50 solved tickets from the last quarter, run them through the agent in a sandbox, and score every response as correct, partly correct, or wrong. That exercise takes about an hour and tells you more than a week of internal debate.
Watch for four failure modes: confidently quoting a policy you retired two years ago, meeting a frustrated customer with a chirpy tone, looping into a second attempt at the same failed answer, and answering a question nobody asked because it spotted a keyword. Fix the instruction, not the individual reply, because the same gap will resurface somewhere else next week.
Measure Resolution Quality, Not Just Deflection
Containment rate is the number everyone reports and the easiest one to game, since a ticket a customer simply abandoned also counts as contained. Track these instead:
- Resolution rate on AI-handled tickets, confirmed by the customer
- CSAT for AI sessions versus human sessions
- Repeat contact rate within seven days on the same issue
- Handle time on tickets the agent escalated
- Escalation accuracy: how many handoffs genuinely needed a person
If repeat contact climbs while containment looks fantastic, the agent is closing tickets rather than solving problems. Platforms such as Ada AI report a similar metric set, and the lesson holds across vendors: resolution quality beats raw automation volume every time.
Add One Intent a Week, Not Ten
The temptation after a good first month is to automate everything at once. Slow expansion is what keeps quality steady. Each week, review the top five reasons tickets were escalated, pick the one with the cleanest answer, write the knowledge article, add the intent, and run it against 20 historical tickets before it goes live.
At that pace you add roughly a dozen intents a quarter without breaking what already works. Enterprise teams moving into voice and multilingual routing usually graduate to a dedicated platform eventually, and the design principles in this guide to building virtual agents with Cognigy.AI map almost one to one onto Zendesk’s configuration screens.
The Weekly Routine That Keeps It Honest
Block 30 minutes every Friday and bring four inputs: the week’s escalation reasons, three randomly chosen AI transcripts, CSAT comments, and any policy change the business shipped. Then make three edits. Update one knowledge article. Tighten one line of the agent’s instructions. Adjust one escalation trigger.
Do that for a quarter and the numbers usually land somewhere sensible: the agent handling 35 to 45 percent of incoming volume, CSAT within three to five points of your human baseline, and nearly every handoff arriving at a human with a summary already attached. That is what a working Zendesk AI agent looks like. Not a bot that sounds convincingly human, but one whose limits you know precisely, because you were the person who drew them.