ExplainabilityTransparencyAccountabilitycs.CV
Poly-CAM: High resolution class activation map for convolutional neural networks
Alexandre Englebert, Olivier Cornu, Christophe De Vleeschouwer
April 28, 2022
Machine Vision and Applications
34 citations
2 influential
Key Takeaways
-
Current explainability methods have significant limitations: Existing techniques for visualizing what neural networks "see" suffer from problems like low resolution (CAM), excessive smoothing (perturbation-based methods), or scattered noise patterns (gradient-based approaches), making it difficult to pinpoint exactly which image features drive predictions.
-
Poly-CAM improves localization precision: The proposed method combines information from both early and late layers of neural networks to create higher-resolution activation maps that more accurately identify which specific image features the network uses to make its decisions.
-
Better feature localization matters for trust: Precise identification of class-specific features is important for responsible AI because it allows humans to verify whether models are making decisions based on relevant, meaningful image features rather than spurious correlations or artifacts.
-
Performance validated on standard metrics: The approach maintains competitive performance on "insertion-deletion faithfulness metrics" (standard tests measuring whether highlighted features actually matter to the model's prediction) while achieving superior precision in locating features.
-
Explainability remains an active challenge: This work demonstrates that making AI systems interpretable requires ongoing technical innovation, as no single existing method adequately balances resolution, smoothness, and accuracy in explaining neural network decisions.