ExplainabilityTransparencyAccountabilitycs.CV
MiSuRe is all you need to explain your image segmentation
Syed Nouman Hasany, Fabrice Mériaudeau, Caroline Petitjean
June 18, 2024
arXiv.org
5 citations
Key Takeaways
-
Explainability gap in AI systems: While deep learning models excel at image segmentation tasks, their decision-making processes remain largely opaque due to their complex, non-linear nature—highlighting a critical challenge in deploying AI systems responsibly across industries.
-
New tool for understanding segmentation decisions: MiSuRe introduces a method to generate "saliency maps" that identify which specific regions of an image a segmentation model actually uses to make decisions, filling a gap since most explainability research has focused on simpler image classification rather than the more complex segmentation task.
-
Practical applications beyond transparency: The paper demonstrates that these saliency maps can serve a dual purpose—not only explaining model decisions but also assessing the reliability of segmentation models after deployment, suggesting explainability tools can have concrete safety benefits.
-
Broad relevance across domains: Testing on three different datasets (artificial, natural images, and medical images) suggests this approach could be valuable across multiple sectors, particularly in high-stakes fields like medical imaging where understanding why a model made a specific segmentation decision is critical for clinical trust and safety.
Abstract
The last decade of computer vision has been dominated by Deep Learning architectures, thanks to their unparalleled success. Their performance, however, often comes at the cost of explainability owing to their highly non-linear nature. Consequently, a parallel field of eXplainable Artificial Intelligence (XAI) has developed with the aim of generating insights regarding the decision making process of deep learning models. An important problem in XAI is that of the generation of saliency maps. These are regions in an input image which contributed most towards the model's final decision. Most work in this regard, however, has been focused on image classification, and image segmentation - despite being a ubiquitous task - has not received the same attention. In the present work, we propose MiSuRe (Minimally Sufficient Region) as an algorithm to generate saliency maps for image segmentation. The goal of the saliency maps generated by MiSuRe is to get rid of irrelevant regions, and only highlight those regions in the input image which are crucial to the image segmentation decision. We perform our analysis on 3 datasets: Triangle (artificially constructed), COCO-2017 (natural images), and the Synapse multi-organ (medical images). Additionally, we identify a potential usecase of these post-hoc saliency maps in order to perform post-hoc reliability of the segmentation model.