ExplainabilityTransparencyAccountabilitycs.CV
Explaining Convolutional Neural Networks through Attribution-Based Input Sampling and Block-Wise Feature Aggregation
Sam Sattarzadeh, Mahesh Sudhakar, Anthony Lem, Shervin Mehryar, K. N. Plataniotis, Jongseong Jang, Hyunwoo Kim, Yeonjeong Jeong, Sangmin Lee, Kyunghoon Bae
October 1, 2020
AAAI Conference on Artificial Intelligence
38 citations
3 influential
Key Takeaways
-
Multi-layer approach improves explanation clarity: The paper demonstrates that combining visualization maps from multiple layers of a CNN produces higher-resolution and clearer explanation maps compared to single-layer methods, addressing a key limitation where previous attribution techniques produced blurry explanations that were difficult to interpret.
-
Systematic layer selection enhances interpretability: The researchers developed a layer selection strategy applicable across different CNN architectures, focusing on the final layers of each convolutional block. This systematic approach provides more consistent and reliable visual explanations regardless of the specific model being used.
-
Method works across diverse real-world applications: The proposed technique was validated on both shallow and deep neural networks using natural images and industrial datasets, suggesting the approach is robust and practical for various AI applications beyond just academic benchmarks.
-
Improved performance on standard evaluation metrics: The method met or outperformed existing state-of-the-art explanation techniques using both ground-truth and model-truth based evaluation metrics, indicating measurable improvements in both explanation ability and visual quality.
-
Stability across different object sizes: The paper notes that the method demonstrates stability regardless of object size variations, which is important for real-world deployment where AI systems encounter objects of varying scales and may need to provide consistent explanations.