ExplainabilityTransparencyAccountabilitycs.LG
Towards Best Practice in Explaining Neural Network Decisions with LRP
Maximilian Kohlbrenner, Alexander Bauer, Shinichi Nakajima, Alexander Binder, Wojciech Samek, Sebastian Lapuschkin
October 22, 2019
IEEE International Joint Conference on Neural Network
183 citations
10 influential
Key Takeaways
-
LRP is a practical tool for understanding neural networks: Layer-wise Relevance Propagation (LRP) is a widely-used method in explainable AI that helps make neural network decisions more transparent by tracing which parts of an input (like pixels in an image) most influenced the model's prediction.
-
Best practices for LRP have now been validated scientifically: The paper provides the first quantitative evidence that the "layer-dependent approach" to LRP—a method that has been used in practice but never rigorously tested—actually does improve how well the explanations represent what the model is really doing.
-
Better explanations lead to more reliable AI: When LRP explanations are improved using the layer-dependent approach, they better identify which objects in images the model is focusing on and more clearly distinguish between different classes, meaning the explanations are more trustworthy for users relying on them.
-
Standardizing XAI methods matters for responsible AI: This research demonstrates the importance of moving beyond informal "best practices" to formally testing and validating explanation methods, which is essential for building confidence in AI systems used in high-stakes applications.
Abstract
Within the last decade, neural network based predictors have demonstrated impressive - and at times super-human - capabilities. This performance is often paid for with an intransparent prediction process and thus has sparked numerous contributions in the novel field of explainable artificial intelligence (XAI). In this paper, we focus on a popular and widely used method of XAI, the Layer-wise Relevance Propagation (LRP). Since its initial proposition LRP has evolved as a method, and a best practice for applying the method has tacitly emerged, based however on humanly observed evidence alone. In this paper we investigate - and for the first time quantify - the effect of this current best practice on feedforward neural networks in a visual object detection setting. The results verify that the layer-dependent approach to LRP applied in recent literature better represents the model's reasoning, and at the same time increases the object localization and class discriminativity of LRP.