ExplainabilityTransparencySafetycs.RO
Approximating a deep reinforcement learning docking agent using linear model trees
Vilde B. Gjærum, Ella-Lovise H. Rørvik, Anastasios M. Lekkas
March 1, 2022
European Control Conference
8 citations
Key Takeaways
-
Transparency through simplification: The researchers successfully approximated a complex deep neural network used for autonomous ship docking with a linear model tree (LMT), which directly connects specific input features to control outputs in an understandable way—addressing a key barrier to deploying AI in safety-critical applications like autonomous vehicles.
-
Real-time explainability is achievable: Unlike existing explanation methods (SHAP and LIME) that are either too computationally expensive or only provide local insights, LMTs can generate real-time explanations of AI decisions, which is essential for autonomous systems that must operate continuously and safely.
-
Practical path to responsible deployment: The paper demonstrates that running an LMT in parallel with the original neural network creates a "digital assurance framework" for autonomous ships—meaning operators can verify the AI's reasoning in real-time without sacrificing the performance of the original trained system.
-
Explainability and performance need not conflict: The approach maintains the effectiveness of the original deep reinforcement learning policy while adding transparency, suggesting that responsible AI doesn't require choosing between accuracy and interpretability in autonomous systems.
-
Domain-specific application matters: The focus on autonomous ship docking illustrates that explainability solutions must be tailored to real-world constraints—in this case, the need for lightweight, fast explanations in maritime operations where safety and regulatory assurance are paramount.