When Medical AI Explanations Help and When They Harm

Explanations can backfire when AI makes mistakes: While AI explanations improved diagnostic accuracy by 6.3 percentage points when the algorithm was correct, they decreased accuracy by 4.9 percentage points when the algorithm was wrong—a critical finding for any field deploying AI decision-support tools.

AI explanations are equally convincing whether right or wrong: The study reveals that modern AI systems generate persuasive-sounding explanations regardless of whether their recommendations are accurate, making it nearly impossible for physicians to tell good advice from bad advice based on the explanation alone.

Doctors significantly overestimate AI accuracy when given explanations: Physicians treated explained AI recommendations as 15.2 percentage points more accurate than they actually were, suggesting explanations create a false sense of confidence rather than genuine understanding.

The most competent doctors suffer the most harm: Experienced physicians with appropriate skepticism were hurt most by misleading explanations (losing 12.4 percentage points in accuracy), while overconfident novices actually benefited slightly—highlighting how transparency can paradoxically harm expert judgment.

Selective transparency outperforms mandatory transparency: The research suggests that strategically showing explanations only when AI is likely correct could generate $2.59 billion in annual healthcare value—43% more than requiring explanations for all AI recommendations, indicating that "more transparency" isn't always better policy.