ExplainabilitySafetyTransparencycs.RO
XAI-N: Sensor-based Robot Navigation using Expert Policies and Decision Trees
Aaron M. Roth, Jing Liang, Dinesh Manocha
April 22, 2021
IEEE/RJS International Conference on Intelligent RObots and Systems
14 citations
2 influential
Key Takeaways
-
Decision trees make AI decisions more understandable: The paper converts a complex deep learning model into a decision tree format, which can be analyzed and modified by humans. This addresses a key responsible AI concern—the "black box" problem—by creating a system where navigation decisions can be traced and understood.
-
Interpretability enables safety improvements without retraining: By extracting the policy into a decision tree, researchers identified and fixed specific problems (like oscillating movements or getting stuck) directly in the decision logic, rather than requiring expensive retraining. This demonstrates how explainability can support safer AI systems.
-
Combining learning with human oversight improves reliability: The approach pairs deep reinforcement learning's adaptability with domain-specific algorithms and human analysis. This hybrid method addresses a critical responsible AI principle: using AI's strengths while maintaining human oversight and control over important decisions.
-
Real-world robot testing validates safety claims: The researchers tested their navigation system with an actual robot moving among pedestrians, not just in simulation. This real-world validation is essential for responsible AI deployment, as it reveals whether safety improvements identified in controlled settings actually work in practice.