ExplainabilityTransparencyTrustworthinesscs.LG
Explainability in Deep Reinforcement Learning, a Review into Current Methods and Applications
Thomas Hickling, Abdelhafid Zenati, Nabil Aouf, Phillippa Spencer
July 5, 2022
ACM Computing Surveys
61 citations
1 influential
Key Takeaways
-
Deep Reinforcement Learning lacks interpretability by default: Since DRL's introduction in 2015, the technology has seen dramatic increases in use across applications, but this growth has been hindered by a fundamental problem—these systems operate as "black boxes" that researchers and the public struggle to understand and trust.
-
Explainable AI (XAI) offers multiple approaches to transparency: Rather than a single solution, the field has developed a variety of methods to open these black boxes, ranging from interpretable symbolic Decision Trees to numerical approaches like Shapley Values, giving developers options suited to different needs.
-
Method-to-application matching is critical: The review identifies that different explainability methods work better for different applications, and some promising techniques may be underutilized in practice—suggesting that choosing the right explanation method for a specific use case matters significantly for building trustworthy AI systems.
-
Trust and understanding are barriers to DRL adoption: The lack of interpretability in Deep Reinforcement Learning has created a credibility gap that prevents wider acceptance of these solutions, indicating that explainability isn't just a technical concern but a practical requirement for responsible deployment.
Abstract
The use of Deep Reinforcement Learning (DRL) schemes has increased dramatically since their first introduction in 2015. Though uses in many different applications are being found, they still have a problem with the lack of interpretability. This has bread a lack of understanding and trust in the use of DRL solutions from researchers and the general public. To solve this problem, the field of Explainable Artificial Intelligence (XAI) has emerged. This entails a variety of different methods that look to open the DRL black boxes, ranging from the use of interpretable symbolic Decision Trees (DT) to numerical methods like Shapley Values. This review looks at which methods are being used and for which applications. This is done to identify which models are the best suited to each application or if a method is being underutilised.