AI2030 Responsible AI Tools Marketplace

Key Takeaways

Saliency maps don't automatically improve human performance: Despite being designed to explain AI decisions, a systematic review of 68 user studies found that saliency maps produce null effects or actually harm performance in many cases, challenging the assumption that more explanation always helps.
The type of task matters significantly: When humans focused on understanding the AI system itself (AI-focused tasks), saliency maps were more beneficial than when humans focused on classifying images directly (image-focused tasks), suggesting explainability tools work better for different purposes than intended.
Saliency maps help in different situations depending on the task: For AI-focused tasks, saliency maps benefited humans most when the AI made incorrect predictions, but for image-focused tasks, they helped most when the AI was correct—indicating that one-size-fits-all explanations may not be optimal.
The specific explanation method used had surprisingly little impact: Contrary to expectations, which particular saliency map technique was employed made little difference to outcomes, suggesting that how explanations are presented to humans may matter more than the technical approach.
More research is needed on real-world factors: The review found limited evidence about how image characteristics and human expertise actually influence whether explanations help, indicating that current user studies may not adequately reflect practical deployment scenarios.

Abstract

Saliency maps can explain how deep neural networks classify images. But are they actually useful for humans? The present systematic review of 68 user studies found that while saliency maps can enhance human performance, null effects or even costs are quite common. To investigate what modulates these effects, the empirical outcomes were organised along several factors related to the human tasks, AI performance, XAI methods, images to be classified, human participants and comparison conditions. In image-focused tasks, benefits were less common than in AI-focused tasks, but the effects depended on the specific cognitive requirements. Moreover, benefits were usually restricted to incorrect AI predictions in AI-focused tasks but to correct ones in image-focused tasks. XAI-related factors had surprisingly little impact. The evidence was limited for image- and human-related factors and the effects were highly dependent on the comparison conditions. These findings may support the design of future user studies.

Key Takeaways

Saliency maps don't automatically improve human performance: Despite being designed to explain AI decisions, a systematic review of 68 user studies found that saliency maps produce null effects or actually harm performance in many cases, challenging the assumption that more explanation always helps.
The type of task matters significantly: When humans focused on understanding the AI system itself (AI-focused tasks), saliency maps were more beneficial than when humans focused on classifying images directly (image-focused tasks), suggesting explainability tools work better for different purposes than intended.
Saliency maps help in different situations depending on the task: For AI-focused tasks, saliency maps benefited humans most when the AI made incorrect predictions, but for image-focused tasks, they helped most when the AI was correct—indicating that one-size-fits-all explanations may not be optimal.
The specific explanation method used had surprisingly little impact: Contrary to expectations, which particular saliency map technique was employed made little difference to outcomes, suggesting that how explanations are presented to humans may matter more than the technical approach.
More research is needed on real-world factors: The review found limited evidence about how image characteristics and human expertise actually influence whether explanations help, indicating that current user studies may not adequately reflect practical deployment scenarios.

Abstract

How explainable AI affects human performance: A systematic review of the behavioural consequences of saliency maps

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links

How explainable AI affects human performance: A systematic review of the behavioural consequences of saliency maps

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links