ExplainabilityTrustworthinessTransparencycs.AI
Explainable Artificial Intelligence (XAI) for Increasing User Trust in Deep Reinforcement Learning Driven Autonomous Systems
Jeff Druce, Michael Harradon, James Tittle
June 7, 2021
arXiv.org
26 citations
2 influential
Key Takeaways
-
Explanations significantly increase user trust in AI systems: The study found statistically significant improvements in how much users trusted and accepted an AI system when it provided explanations compared to when it operated as a "black box," suggesting that transparency is a practical tool for responsible AI deployment.
-
Multi-layered explanations are more effective than single explanations: The framework combined three types of information—visual performance data, generalization across similar scenarios, and narrative summaries—indicating that users benefit from understanding AI decisions from multiple angles rather than relying on one explanation type alone.
-
Users need to know when AI systems might fail: A key focus of the research was helping users understand "when their output can be trusted," highlighting that responsible AI isn't just about explaining what a system does, but specifically about communicating its limitations and reliability in different situations.
-
Visual and narrative explanations work together: The combination of graphical depictions with plain-language narrative explanations proved effective, suggesting that responsible AI communication should meet users where they are—some people understand data visualizations better, while others prefer written explanations.
-
Human evaluation is essential for XAI design: Rather than assuming explanations would help, the researchers conducted actual user experiments to test their framework, demonstrating that responsible AI requires validating that explanations genuinely improve human understanding and trust, not just theoretically.