ExplainabilitySafetyTrustworthinesscs.AI
Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery
Devleena Das, Siddhartha Banerjee, Sonia Chernova
January 5, 2021
IEEE/ACM International Conference on Human-Robot Interaction
153 citations
11 influential
Key Takeaways
-
Non-experts need context and history to understand robot failures: The research found that explanations are most helpful to everyday users when they include information about what led up to the failure and the sequence of actions the robot attempted, rather than just technical details about system components.
-
Current AI explanations are designed for experts, not users: Most existing explainable AI techniques target domain specialists, but when robots fail in homes or offices, they often need help from regular people who lack technical training—highlighting a gap in how AI systems communicate with the general public.
-
Robots can automatically generate helpful failure explanations: The researchers successfully developed a model that can autonomously create explanations for why a robot failed at a task (specifically pick-and-place operations), moving beyond hand-written explanations toward scalable solutions.
-
Generated explanations work across different environments: The study demonstrated that automatically-generated explanations created for one setting (a home environment) were equally effective when applied to an unseen office environment, suggesting these explanations can generalize and be reused.
-
Robot failures are inevitable in real-world settings: The paper emphasizes that as robots become more integrated into everyday life, occasional failures are unavoidable—making it crucial to design systems that help non-expert users understand and assist with recovery rather than just preventing failures.