ExplainabilityTransparencyTrustworthinesscs.HC
Explainable AI improves task performance in human-AI collaboration
Julian Senoner, Simon Schallmoser, Bernhard Kratzwald, Stefan Feuerriegel, Torbjørn Netland
June 12, 2024
Scientific Reports
113 citations
3 influential
Key Takeaways
-
Explainable AI enables better human judgment: When AI systems showed how they reached their conclusions through visual explanations (heatmaps), domain experts performed better than when using black-box AI. This suggests that transparency allows professionals to validate AI recommendations against their own expertise rather than blindly trusting algorithmic outputs.
-
Real-world validation across multiple domains: The study tested this finding with two different professional groups performing authentic tasks—factory workers inspecting 9,600 electronic products and radiologists analyzing 5,650 chest X-rays. The consistent improvement across manufacturing and medical contexts suggests the benefits of explainable AI extend broadly, not just in laboratory settings.
-
Explainability supports appropriate AI reliance: The research indicates that explainable AI helps humans know when to trust AI predictions (when accurate) and when to override them (when wrong). This is critical for responsible AI deployment, as it prevents both over-reliance on flawed algorithms and under-utilization of accurate AI assistance.
-
Transparency is essential for professional accountability: In high-stakes fields like medicine and manufacturing, professionals need to understand AI reasoning to maintain responsibility for their decisions. Explainable AI preserves human agency and accountability rather than creating a "black box" that professionals must blindly follow.