ExplainabilityTransparencyAccountabilitycs.LG
Effect of Superpixel Aggregation on Explanations in LIME -- A Case Study with Biological Data
Ludwig Schallner, Johannes Rabold, Oliver Scholz, Ute Schmid
October 17, 2019
PKDD/ECML Workshops
30 citations
2 influential
Key Takeaways
-
Explanation tools are sensitive to technical choices: LIME, a popular method for explaining AI decisions on images, produces significantly different results depending on which segmentation algorithm is used—meaning the "explanation" you get can depend more on the tool's settings than on what the AI actually learned.
-
Not all explanation methods are equally reliable: The study found that Quick-Shift (LIME's default algorithm) showed the least agreement with human-identified important image regions, while Compact-Watershed performed best, suggesting that default settings in explanation tools may not be optimal.
-
Visual explanations require validation: Since different segmentation methods produced "strongly varying" relevance areas, this highlights the importance of validating explanation outputs against human judgment rather than accepting them at face value—a critical consideration when using these tools to build trust in AI systems.
-
Implications for high-stakes domains: In biological image analysis and other sensitive applications, the choice of explanation method could influence which features are highlighted as important to decision-makers, potentially affecting how much confidence stakeholders place in an AI system's recommendations.
Abstract
End-to-end learning with deep neural networks, such as convolutional neural networks (CNNs), has been demonstrated to be very successful for different tasks of image classification. To make decisions of black-box approaches transparent, different solutions have been proposed. LIME is an approach to explainable AI relying on segmenting images into superpixels based on the Quick-Shift algorithm. In this paper, we present an explorative study of how different superpixel methods, namely Felzenszwalb, SLIC and Compact-Watershed, impact the generated visual explanations. We compare the resulting relevance areas with the image parts marked by a human reference. Results show that image parts selected as relevant strongly vary depending on the applied method. Quick-Shift resulted in the least and Compact-Watershed in the highest correspondence with the reference relevance areas.