Painting the black box white: experimental findings from applying XAI to an ECG reading setting

XAI doesn't automatically improve understanding: The paper warns that explainable AI approaches risk "painting the black boxes white"—creating an illusion of transparency without actually making AI systems more understandable or usable to decision-makers, potentially introducing new errors in the process.

Expertise matters for how doctors perceive AI explanations: The study of 44 cardiology residents and specialists found that users' level of expertise significantly influenced how they perceived AI explanations' quality and their trust in the system, suggesting that one-size-fits-all XAI solutions may not work across different skill levels.

Better explanations can increase over-reliance on AI: The research identified a "white-box paradox" where improved explanations may paradoxically increase users' tendency to defer decisions to automation (technology dominance) rather than maintaining appropriate human judgment in medical decision-making.

Cognitive factors are critical to XAI evaluation: The paper emphasizes that assessing AI systems requires examining the cognitive dimension of how users actually perceive and process explanations, not just whether explanations are technically transparent—a human-centered rather than purely technical approach.

Current XAI methods need human-centered redesign: The findings suggest that simply making AI decision-making processes visible to doctors isn't enough; XAI systems must be specifically designed with consideration for how medical professionals actually think and make decisions.