Challenging common interpretability assumptions in feature attribution explanations

Feature attribution explanations may not improve decision-making as assumed: The study's large-scale human-subjects experiment found that feature attribution explanations (a common XAI method) provided only "marginal utility" for human decision makers, challenging the widespread assumption that these explanations automatically help people make better choices.

Explanations can sometimes make decisions worse: In certain cases, the study found that providing feature attribution explanations actually resulted in worse decisions compared to no explanation, suggesting that the cognitive burden or misleading nature of explanations can have unintended negative consequences in real-world settings.

"Placebo explanations" revealed cognitive confounders: By testing a control group that received fake explanations, researchers demonstrated that cognitive and contextual factors—not just the quality of the explanation itself—influence how people use AI systems, indicating that human psychology plays a larger role than previously acknowledged in XAI research.

Current XAI evaluation methods are insufficient: The paper critiques the field's reliance on offline simulations and proxy metrics (like model complexity) rather than actual human evaluation, arguing that these indirect measures fail to capture whether explanations truly help people in realistic decision-making scenarios.

Human evaluation should be standard in XAI research: The authors emphasize that explainability should not be assumed but rigorously tested with real people in realistic contexts before deploying these systems in high-stakes applications like healthcare, criminal justice, or financial decisions.