ExplainabilityTransparencyTrustworthinesscs.LG
A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks
Atefeh Shahroudnejad
February 2, 2021
arXiv.org
43 citations
1 influential
Key Takeaways
-
Deep neural networks lack transparency in high-stakes decisions: The paper emphasizes that DNNs are often treated as "black-box systems" that don't reveal their internal reasoning, which is particularly problematic when these systems make critical decisions affecting human lives, such as in medical applications and control systems.
-
Explainability is essential but underdeveloped: The survey identifies explainable artificial intelligence (XAI) as still being "in its infancy," indicating that despite recent efforts to explain DNN behavior, the field lacks mature, standardized approaches for making these systems interpretable to users and stakeholders.
-
Understanding DNNs requires multiple approaches: The paper advocates for a comprehensive approach combining three complementary methods—understanding, visualization, and explanation—to reveal both the internal processing actions and overall behavior of deep networks, suggesting no single method is sufficient.
-
Trust depends on explainability: The paper frames explanation as fundamental to building trust in deep learning systems, arguing that understanding "the argument behind deep models' decisions" is paramount when these systems are deployed in applications where human welfare is at stake.
Abstract
Recent advancements in machine learning and signal processing domains have resulted in an extensive surge of interest in Deep Neural Networks (DNNs) due to their unprecedented performance and high accuracy for different and challenging problems of significant engineering importance. However, when such deep learning architectures are utilized for making critical decisions such as the ones that involve human lives (e.g., in control systems and medical applications), it is of paramount importance to understand, trust, and in one word "explain" the argument behind deep models' decisions. In many applications, artificial neural networks (including DNNs) are considered as black-box systems, which do not provide sufficient clue on their internal processing actions. Although some recent efforts have been initiated to explain the behaviors and decisions of deep networks, explainable artificial intelligence (XAI) domain, which aims at reasoning about the behavior and decisions of DNNs, is still in its infancy. The aim of this paper is to provide a comprehensive overview on Understanding, Visualization, and Explanation of the internal and overall behavior of DNNs.