ExplainabilitySafetyTrustworthinesscs.AI
Explainable AI for System Failures: Generating Explanations that Improve Human Assistance in Fault Recovery
Devleena Das, Siddhartha Banerjee, Sonia Chernova
November 18, 2020
arXiv.org
7 citations
Key Takeaways
-
Clear explanations of AI failures help non-experts assist in recovery: The research demonstrates that when AI systems fail, providing natural language explanations of what went wrong enables ordinary users without technical expertise to better understand the problem and take appropriate corrective action, rather than being left confused or helpless.
-
Context matters more than just identifying the failure: The paper introduces "context-based" explanations that go beyond simply naming what failed—they explain the surrounding circumstances and why the failure occurred in that specific situation, which the authors found helps users both understand causes and select proper recovery strategies.
-
AI can be trained to explain its own failures automatically: Rather than requiring engineers to manually write explanations for every possible failure scenario, the researchers developed a machine learning model that can automatically generate contextual explanations across different types of failures and situations, making this approach scalable for real-world deployment.
-
Explainability is essential for safe human-AI collaboration: As robots and AI systems become more integrated into everyday environments, the ability to clearly communicate failures back to users is critical for safety and trust—users cannot effectively help recover from problems they don't understand.