Evaluating Saliency Map Explanations for Convolutional Neural Networks: A User Study

Saliency maps have limited practical utility: While saliency map explanations (specifically using the LRP algorithm) helped users understand which image features the CNN focused on, they provided "very limited help" for users to predict how the system would behave on new images—suggesting current explanation methods may not translate to actionable understanding.

Explanation algorithms need rigorous user testing: The paper emphasizes that despite various explanation algorithms being proposed for CNNs, "limited research effort" has evaluated whether these explanations actually help people understand AI systems, highlighting a critical gap between technical development and real-world usability.

Instance-level explanations alone are insufficient: The authors argue that the AI and human-computer interaction communities should "look beyond instance-level explanations" (explanations for individual predictions), suggesting that helping people understand AI systems requires broader, more comprehensive approaches than current methods provide.

Expert-level interpretability doesn't guarantee user understanding: Even though CNNs are "hard to interpret, even for experts," the study shows that providing visual explanations doesn't automatically make the systems more understandable to general users, indicating a disconnect between technical interpretability and practical comprehension.