Six weeks into a pilot, a mid-sized insurer had Amelia AI handling 38% of its claims-status chats without a human touching them. Then a customer typed: “my dad passed away last month, is his policy still active?” The agent offered a link to the claims form.
That one exchange explains why rollout design matters more than model choice. Amelia AI reads context, triggers back-office actions and holds a conversation that doesn’t feel like a phone menu. It will also confidently misfire on the small percentage of conversations that carry the most weight, unless someone maps the edges in advance.
Here’s the playbook I’d hand to a support or operations lead starting this month: seven steps, in order, with the decisions you’ll need to make at each one.
1. Choose one process, not a department
The instinct is to point Amelia at “customer service” and let it learn. Resist that. Pick a single process with three properties: high volume, low variance, and a definition of done you could explain to a new hire in one sentence.
Strong first candidates include order status lookups, appointment rescheduling, password resets, balance and policy enquiries, and refunds below a set value. Weak ones include billing disputes, anything involving a complaint or a bereavement, and processes where the answer depends on a judgement call between three teams. You can reach those later, once the plumbing works.
Look for at least 1,500 conversations a month in whichever process you choose. Below that there isn’t enough data to tune intents properly. Bear in mind the bar customers now bring with them, too. After two years of talking to assistants built into their phones, they arrive expecting the assistant features shipping on the next iPhone, not a keyword matcher that replies “I didn’t understand that.”
2. Map the conversation before you open the platform
Spend a week in transcripts instead of in the tool. Two artefacts come out of that week.
Write the happy path in five turns
“Where’s my order?” should run: ask for an order number or email, look up the record, state the status, offer a tracking link, close. Five turns. If the design needs twelve, you’re automating a process nobody has simplified yet, and the agent will inherit every inefficiency.
List the twenty intents behind 80% of volume
Tag 300 recent conversations by hand. Most teams find the long tail is shorter than they feared: three or four phrasings usually cover the bulk of how people actually ask. Those become your intents. Everything else falls back to a human, which is exactly what you want in month one.
3. Train on real transcripts, including the messy ones
A common mistake here is pasting polished help-centre articles into the knowledge base and expecting conversational answers. The agent then sounds like a leaflet, because that’s what it was given.
Feed it 500 to 1,000 real conversations instead. Typos, half-finished sentences, “helo” and “cancell”, voice-to-text garble, three languages in one thread. Amelia’s language layer handles imperfect input well; it just needs input that reflects how people write. The move from rules and keywords to genuine intent understanding is why enterprise agents like Amelia have left old-school chatbots behind, and this step is where that advantage shows up or doesn’t.
Give it a voice your brand can defend
Decide three things and write them down: how formal it sounds, how long its sentences are, and whether it apologises. Then review twenty sample responses against those rules before anything goes live. Support leaders consistently underestimate how much tone drives trust in the first thirty seconds.
4. Wire the actions, not just the answers
Amelia earns its keep when it does things. A “change my delivery address” flow looks like this:
- Verify identity with a one-time code tied to the account, every time
- Read the order state and allow the edit only if it hasn’t shipped
- Write the change, then read the new address back for confirmation
- Log the edit against the ticket so the next agent sees it
Two guard rails matter more than the rest: never let the agent act on an unverified account, and cap the value of any transaction it can approve alone. Make every write idempotent so a retry after a timeout doesn’t issue a second refund. For IT teams, connectors to ServiceNow, Zendesk and Jira cover most of the plumbing out of the box.
5. Set escalation rules before launch, not after
Handoff design decides whether people trust the system. A bot that resolves 40% of chats but traps the other 60% for six minutes before passing them on is worse than no bot at all.
Rules that hold up in production:
- Two consecutive failed intent matches, straight to a human
- Any mention of legal action, bereavement, medical symptoms or a regulator
- Sentiment signals: profanity, capitals, the same question repeated
- Accounts flagged VIP, vulnerable, or already carrying an open complaint
- Any financial action above your approval threshold
- An explicit request for a person, honoured immediately with no attempt to talk them out of it
On handover, pass the summary, the intent it detected and everything already collected. Making a customer repeat their account number to a human is the fastest way to undo the goodwill the automation earned.
Some categories need tighter rules than others. If your product touches health data, such as a fitness band flagging an irregular heart rhythm, a chat about that alert shouldn’t be resolved by an agent at all. Send it to a clinical line.
6. Launch at 5%, measure four numbers, then widen
Go live on one queue, one channel, or a few hours a day. Five per cent of traffic is enough to surface the failures. The four numbers worth a dashboard:
- Containment rate: conversations closed without a human
- False containment: marked resolved, but the customer returns within 48 hours
- Escalation accuracy: handoffs a human agreed were necessary
- CSAT, split: bot-resolved versus human-resolved
Don’t draw conclusions before 1,200 conversations. A realistic path runs 25–35% containment in week two and 55–70% by day 90, after two serious rounds of intent tuning. If false containment sits above 8%, pause expansion and fix the knowledge gaps first.
7. Fund the boring part
Licence fees are the visible cost. The hidden one is maintenance: reviewing unresolved intents, retiring answers that no longer match policy, adding the stragglers from the long tail. Budget four to six hours a week for one person and treat it as permanent, not a launch task.
That matters because budgets are tight and tightening further. The same cost pressure behind why the iPhone is about to get more expensive shows up in every software renewal you’ll sign this year, and a support automation line item that can’t show containment numbers won’t survive the next review.
What the first 90 days should look like
By day 14, one process is live at 5% of traffic, sitting near 30% containment, with 60 transcripts reviewed by hand. Day 30 brings escalation rules adjusted twice and false containment under 10%. Day 60 gets the process above 50% containment with a second one mapped on paper. Day 90 lands between 55% and 70%, depending on how messy the underlying systems turned out to be.
The trigger for scaling is specific. Add the next process only when the first one holds a 60% containment rate for two consecutive weeks with no new fixes. That figure means your transcripts, handoff rules and measurement are all doing their job. It’s a more reliable green light than any vendor benchmark, and it’s the one worth waiting for.