Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Setting

Text explanations alone can dangerously increase over-reliance on AI: The study found that when healthcare practitioners received natural language explanations, they were significantly more likely to trust the AI's recommendations, even when those recommendations were incorrect—a critical safety concern in medical settings.

Combining visual and text explanations provides better safeguards: Pairing saliency maps (visual highlights showing where the AI focused) with text explanations reduced over-reliance and helped practitioners maintain appropriate skepticism, suggesting that multiple explanation types work together to support safer decision-making.

Explanation quality matters as much as explanation type: The study found that explanations containing factually correct information that aligned with the AI's actual performance were significantly more useful than explanations that didn't match reality, highlighting that poorly designed explanations can actively harm rather than help.

Real-world testing reveals gaps in current XAI methods: By studying actual healthcare practitioners rather than just testing explanations in isolation, the researchers discovered that current explainability approaches don't work as intended in practice—a finding that challenges assumptions about how explanations improve AI safety.

Explainability alone doesn't guarantee responsible AI use: The research demonstrates that simply making AI "more transparent" doesn't automatically make it safer; the design and accuracy of explanations fundamentally shape whether they help or harm human decision-making in critical domains.