Semantic-Based Explainable AI: Leveraging Semantic Scene Graphs and Pairwise Ranking to Explain Robot Failures

Automated explanations outperform manual ones: The framework's automatically generated semantic explanations helped everyday users better understand and respond to robot failures compared to hand-annotated explanations, suggesting that scalable AI explanation systems can be more effective than labor-intensive manual approaches.

Semantic scene understanding enables better explanations: By automatically extracting spatial relationships and object attributes from environments using semantic scene graphs, the system can provide contextually grounded explanations without requiring humans to manually annotate each scenario—addressing a major scalability limitation in current approaches.

Non-experts need accessible failure explanations: The research recognizes that everyday people, not just trained technicians, will be first responders to robot failures in homes and communities, highlighting the importance of designing AI explanations that are understandable to general users rather than specialists.

Pairwise ranking focuses explanations on what matters: The use of pairwise ranking helps the system prioritize which semantic information is most relevant to explaining a specific failure, making explanations more focused and useful rather than overwhelming users with all possible contextual details.

Generalizability is critical for real-world deployment: The paper emphasizes that explanation systems must work across different environments and scenarios rather than being customized for individual cases, which is essential for responsible AI systems that need to operate reliably in diverse human settings.