Fraud stopped being a purely human activity sometime in the last year. Not all of it, obviously; I mean, most fraud is still someone typing on a phone somewhere trying to move money that isn’t theirs.
But enough of it has shifted that Experian’s 2026 Future of Fraud Forecast put a name on it. AI agents transacting and shopping on people’s behalf are now a real driver of financial fraud.
The report calls it “machine-to-machine mayhem,” which sounds like a headline written to get clicks, and honestly, it worked.
Because a legitimate shopping agent and a fraudulent bot can show up in a transaction log looking basically identical. Nobody’s figured out the tell yet.
I want to sit with why that’s a bigger deal than it sounds. Most fraud detection, at every scale, from a local fintech to a Fortune 500 bank, has quietly rested on the same assumption for a decade or more.
Which is that there’s a human on the other end of the transaction, and humans leave fingerprints.
Things like how they type, when they switch devices, basically the way someone behaves when they’re anxious or in a hurry.
Every fraud model, whether anyone building it says so out loud, is trained to notice when a human stops acting like themselves.
I ran into a smaller, dumber version of this exact problem while building a fraud detection system, one that leaned hard on SHAP to explain itself.
It’s worth telling because the thing that got me out of trouble at the time is the exact thing I now think is running out of road.
The question that wasn’t really about the model
At my presentation, one of the panelists leaned back in his chair and asked a question that had nothing to do with precision or recall.
If a CEO makes a one-million-dollar transaction, and a university student makes the same transaction, does your system flag them the same way?
I remember the room going quiet, and I remember taking longer to answer than I wanted to.
My model, a Random Forest classifier I’d trained on the PaySim dataset after benchmarking it against XGBoost and LightGBM, didn’t know or care who was transacting.
It looked at the transaction: amount, timing, device, velocity.
Basically the same features for a CEO’s account as for a broke undergrad’s.
On paper, that’s what fairness in ML is supposed to look like. In front of a panel whose entire job is finding holes in risk models, it looked like I’d missed something obvious.
Here’s what actually saved me, and it wasn’t a smarter model.
I’d wired SHAP into the pipeline sitting on top of the classifier, mostly because a supervisor earlier in the year had pushed back on treating the model as a black box, and it turned out to be the reason I survived that question.
I could pull up the specific transaction, show the ranked SHAP values, and say: here’s what actually drove this prediction, in order, and with numbers attached.
That question has stayed with me longer than almost anything else from that presentation. Not because of what it exposed about my model specifically, but because of the assumption sitting underneath the whole field: that the thing making a transaction has a life outside the transaction, a job, a history, a reason to be moving that particular amount of money.
In simpler words, context.
If you want the full build, the model comparisons, the PaySim dataset quirks, and what my supervisor actually said when he saw the SHAP output for the first time, I wrote it all up here: I Trained Six Models for Fraud Detection, and the Best One Isn’t in Production. This piece picks up from that one but doesn’t require you to have read it.
I Trained Six Models for Fraud Detection, and the Best One Isn’t in ProductionWhat a final-year project taught me about the gap between evaluation metrics and production decisionsBenjamin Nweke · 10 min read
···
Why explainability stopped being optional
Black-box fraud models have always had a trust problem, agentic AI or not.
The models that perform best, deep ensembles, gradient-boosted trees, tend to be the hardest to interpret, and that’s a genuinely annoying trade-off if you’ve ever had to explain a flagged transaction to someone who isn’t a data scientist.
If a compliance analyst can’t tell whether a flag is a real signal or just noise the model latched onto, they’ll either rubber-stamp everything or ignore the model entirely.
I’ve seen both happen, and neither one is why you built the thing.
SHAP earned its spot in fraud stacks by answering a narrow, unglamorous question, not “is this model good” but “why did it do this, for this one transaction, right now.”
It’s less exciting than model architecture, which is maybe why it doesn’t come up in conversation as much as it probably should.
A risk score with no reasoning attached is close to useless the moment a human has to act on it. That was my exact problem at the presentation.
Instead of “92% fraud probability,” you get something like: the amount is high for this account, the account is under 48 hours old, or the merchant has prior flags.
People trust that.
Or at least they trust it more than a number with nothing under it.
The ground is shifting under the fingerprint
Here’s the part that bugs me.
Every explainability method built for fraud detection, SHAP included, explains a decision using features that describe a transaction: amount, time, device, velocity.
Now stay with me here.
Those features exist because they correlate with how humans behave when something’s wrong.
A student’s account suddenly moving CEO-sized money looks strange precisely because we assume a stable human behind it, with a pattern of life the anomaly is breaking from.
An agent doesn’t have a pattern of life in any way that maps onto those features.
It doesn’t get tired at 2 am. It doesn’t fat-finger a transfer or panic-send money to the wrong account.
It can transact at a speed and consistency no human could sustain for five minutes, and that breaks the whole premise fraud detection has quietly run on: that deviation from a human baseline is the signal.
This isn’t a someday problem. It’s already showing up in the numbers.
The number I find more interesting than the headline 84% is the 60% who expect AI-mediated banking to make traditional fraud defenses less effective.
That points to something deeper than criminals simply getting better tools. The signals those defenses were built around may themselves be changing.
Regulators are already circling this, and not in the usual glacial way.
NIST’s AI Agent Standards Initiative, launched in February 2026, treats agents as identifiable non-human principals that need their own credential lifecycles and audit trails, on the reasoning that existing auth frameworks were built for humans and don’t stretch to cover autonomous software.
In the US, it’s already further along than a standards initiative. Senator Mark Warner’s draft AI AGENT Act, floated in mid-2026, would create a federal registry of trusted agents and require them to act transparently in a user’s interest on major platforms.
I don’t know if this exact bill survives contact with Congress; most don’t in their first draft.
But the instinct behind it is the thing worth noticing: lawmakers have landed on identity and accountability as the load-bearing question, before the industry has agreed on how to even log an agent’s reasoning in a way a human could audit later.
What breaks, and what doesn’t
I want to be clear: I don’t think SHAP is dead or dying, trust me, not even close.
Feature-level explanations still matter for the transaction-shaped slice of fraud, and humans aren’t going anywhere as fraud actors, unfortunately.
But feature attribution on transaction fields only ever answered why did this look wrong. Agentic fraud is dragging a different question onto the table.
Why did this actor do this at all?
That second question wants a different kind of trace entirely.
Not transaction features but the agent’s actual decision path, what tools it called, what it was authorized to do, and whether the action it took falls inside or three steps outside the scope someone delegated to it.
A system built to explain a Random Forest’s verdict on a dollar amount has literally nothing to say about whether an agent went off-script upstream of the transaction it eventually made.
That’s closer to auditing a trajectory than scoring a data point.
I’ve gone looking for someone who’s solved this properly and mostly found papers that gesture at the problem without actually closing it, which either means I’m bad at finding things or it’s genuinely still open. Possibly both.
(Also, and this is a side thought I can’t fully back up: I wonder if part of the reason nobody’s cracked this yet is that agent trajectories are logged by the same companies building the agents, and there’s not a lot of incentive to make that logging legible to an outside auditor. That’s more of a suspicion than an argument, so take it for what it’s worth.)
My guess, and it’s just a guess, is that this doesn’t get fixed by making SHAP more sophisticated.
It probably gets fixed by borrowing from a completely different corner of AI safety, agent monitoring and trajectory evaluation, and then stitching it onto the explainability discipline fraud detection already has.
Or maybe those two fields never really merge and someone builds a third thing from scratch. I go back and forth on which one is more likely.
···
Final thoughts
I built a system to catch a human doing something a human wouldn’t normally do, and SHAP was the thing that made it defensible in front of a room of people paid to find its weak points.
That problem is still real, and still not fully solved, so this isn’t me saying my fraud detection system or others like it are done for.
But the target has moved somewhere the current toolkit wasn’t initially built to follow, and I don’t think that’s a small footnote.
It’s the gap between explaining a score and explaining an actor. Right now most of us, me included, only really know how to do the first one.
If you’re building or evaluating fraud detection, the question I’d actually sit with isn’t whether your model is accurate.
It’s whether your explanations still mean anything once the thing on the other end of the transaction stops being a person.
···
Before you go!
I write about the gap between what a model does and what you can actually prove it did, especially once the thing making decisions stops being obviously human. You can subscribe to my newsletter if you’d like more of that.
Connect With Me