The Effect of Explainable AI-based Decision Support on Human Task Performance: A Meta-Analysis

XAI explanations do improve task performance, but modestly: The meta-analysis found that explainable AI-based decision support systems do help humans perform better on classification tasks, though the improvement is not as dramatic as some proponents might suggest.

The quality of research matters more than explanation type: The study's most surprising finding is that how well a study was conducted (its "risk of bias") has a bigger impact on whether explanations help than the type of explanation used—suggesting that rigorous testing methods are crucial for understanding XAI's real-world value.

Explanations alone aren't the magic ingredient: While XAI systems improve performance, the explanations themselves are "not the decisive driver" of improvement, indicating that other factors in the decision support system design may be equally or more important than the explanations themselves.

We still don't fully understand human-AI collaboration: The inconsistent findings across different studies highlight that the field lacks a clear understanding of how humans and AI systems should work together effectively, suggesting more research is needed before deploying XAI systems at scale.

Transparency doesn't automatically equal better decisions: This research cautions against assuming that simply making AI more transparent will solve problems—implementation details and study quality matter significantly for whether explanations actually help people make better decisions.