ExplainabilityTransparencyBiascs.CV
Learning Visual Explanations for DCNN-Based Image Classifiers Using an Attention Mechanism
Ioanna Gkartzonika, Nikolaos Gkalelis, Vasileios Mezaris
September 22, 2022
ECCV Workshops
10 citations
Key Takeaways
-
AI systems can be made more transparent without sacrificing performance: The proposed methods (L-CAM-Fm and L-CAM-Img) generate visual explanations for image classification decisions while maintaining competitive accuracy on ImageNet, demonstrating that explainability and performance are not necessarily in conflict.
-
Visual explanations can reveal hidden dataset problems: The paper's qualitative analysis shows that examining what image regions the AI focuses on can uncover dataset biases affecting the classifier, suggesting that explainability methods serve as a diagnostic tool for identifying training data issues that might otherwise go unnoticed.
-
Efficient explanations are practical for real-world deployment: Both methods require only a single forward pass during inference to generate explanations, making them computationally feasible for practical applications rather than adding significant computational overhead.
-
Understanding classification errors requires looking at what the AI "sees": The paper demonstrates that visual explanations provide valuable insight into why classifiers make mistakes, enabling developers and auditors to understand whether errors stem from model limitations, ambiguous images, or problematic training data—information essential for responsible AI deployment.
Abstract
In this paper two new learning-based eXplainable AI (XAI) methods for deep convolutional neural network (DCNN) image classifiers, called L-CAM-Fm and L-CAM-Img, are proposed. Both methods use an attention mechanism that is inserted in the original (frozen) DCNN and is trained to derive class activation maps (CAMs) from the last convolutional layer's feature maps. During training, CAMs are applied to the feature maps (L-CAM-Fm) or the input image (L-CAM-Img) forcing the attention mechanism to learn the image regions explaining the DCNN's outcome. Experimental evaluation on ImageNet shows that the proposed methods achieve competitive results while requiring a single forward pass at the inference stage. Moreover, based on the derived explanations a comprehensive qualitative analysis is performed providing valuable insight for understanding the reasons behind classification errors, including possible dataset biases affecting the trained classifier.