ExplainabilityTransparencyTrustworthinesscs.CV
Explain Any Concept: Segment Anything Meets Concept-Based Explanation
Ao Sun, Pingchuan Ma, Yuanyuan Yuan, Shuai Wang
May 17, 2023
Neural Information Processing Systems
69 citations
Ao Sun, Pingchuan Ma, Yuanyuan Yuan, Shuai Wang
Explanations can now use any visual concept, not just pre-defined ones: Traditional concept-based AI explanation methods required researchers to manually select which concepts (like "fur" or "wheels") to look for in advance. This paper's method, EXplain Any Concept (EAC), automatically identifies relevant visual segments in images, making explanations more flexible and potentially more relevant to what the AI actually "sees."
Visual explanations are moving beyond highlighting individual pixels: Pixel-based explanation methods—which show which parts of an image influenced a decision—are difficult for humans to interpret and can be unreliable. This work demonstrates that grouping pixels into meaningful visual concepts (like "a dog's head" rather than scattered pixels) produces clearer, more interpretable explanations of AI decisions.
Efficiency matters for practical AI transparency: While the underlying segmentation technology (Segment Anything Model) is powerful, it's computationally expensive to run repeatedly. The researchers developed a "lightweight" alternative that maintains explanation quality while reducing computational costs, addressing a real barrier to deploying explainability tools in practice.
The method was validated on real-world image datasets: Testing on ImageNet and COCO—standard benchmarks representing diverse real-world images—shows the approach works across varied visual scenarios, suggesting potential applicability beyond controlled laboratory settings.
This bridges a gap between human-interpretable and automated explanations: By combining automatic visual segmentation with concept-based explanations, the method reduces the need for manual human annotation while still producing explanations in terms humans naturally understand.
EXplainable AI (XAI) is an essential topic to improve human understanding of deep neural networks (DNNs) given their black-box internals. For computer vision tasks, mainstream pixel-based XAI methods explain DNN decisions by identifying important pixels, and emerging concept-based XAI explore forming explanations with concepts (e.g., a head in an image). However, pixels are generally hard to interpret and sensitive to the imprecision of XAI methods, whereas "concepts" in prior works require human annotation or are limited to pre-defined concept sets. On the other hand, driven by large-scale pre-training, Segment Anything Model (SAM) has been demonstrated as a powerful and promotable framework for performing precise and comprehensive instance segmentation, enabling automatic preparation of concept sets from a given image. This paper for the first time explores using SAM to augment concept-based XAI. We offer an effective and flexible concept-based explanation method, namely Explain Any Concept (EAC), which explains DNN decisions with any concept. While SAM is highly effective and offers an "out-of-the-box" instance segmentation, it is costly when being integrated into defacto XAI pipelines. We thus propose a lightweight per-input equivalent (PIE) scheme, enabling efficient explanation with a surrogate model. Our evaluation over two popular datasets (ImageNet and COCO) illustrate the highly encouraging performance of EAC over commonly-used XAI methods.
Citation data from Semantic Scholar