“It does help to know a little bit about how these systems work so that you can understand better when an algorithm might be leading you astray,” Langlotz says.
Whenever a human or machine makes a yes/no error in the interpretation of a medical image, there are two possible ways to go wrong: A false negative misses the presence of disease, whereas a false positive sees one when it isn’t actually there.
For uncommon diseases—and most diseases are relatively uncommon—any diagnostic will tend to produce a lot of false positives simply because it encounters large numbers of people without the disease, giving it more opportunities to issue false alarms. A physician needs to be able to recognize and overrule these false positives while still correctly identifying the machine’s relatively rare correct diagnosis. Believing too much in the machine’s results—positive or negative—is called automation bias.
But it’s important to avoid accepting false negatives, too. For example, an AI system might miss the presence of blood in the brain on a CT or MRI scan when it’s actually there. Trusting in such oversights is called automation complacency.
“These are both issues of letting the AI change the radiologist’s level of suspicion without realizing it,” Kottler says. Over time, people may start to believe AI’s answers too much. One study found that even experienced radiologists saw big drops in the accuracy of their mammography interpretation when their decisions were made under the influence of incorrect AI predictions.
On the other hand, the issue of AI distrust is a bit more intuitive. Many people will naturally be suspicious of a new technology—especially one that has been accused of potentially taking their jobs—until it proves its worth. And it doesn’t help that otherwise reliable AI systems make obvious mistakes that are different from human errors. “So you see a silly mistake, a silly false positive, and you’re like, ‘Well, of course the AI is not smart,’ and you could then dismiss it,” Kottler says.