Explainable AI (XAI) in Image Segmentation in Medicine, Industry, and Beyond: A Survey

Semantic segmentation explainability has been overlooked compared to image classification, despite widespread deployment in critical domains like medical imaging and industrial applications, indicating a significant gap in responsible AI practices for these high-stakes uses.

The field lacks standardized evaluation metrics and datasets for assessing explainability in segmentation tasks, making it difficult to compare different explanation methods and establish best practices for responsible deployment.

A formal taxonomy for interpretable semantic segmentation is needed to organize and advance the field systematically, suggesting that the responsible AI community should establish clearer frameworks for understanding and validating explanation techniques in segmentation.

Medical and industrial applications require urgent attention to explainability, as these domains involve consequential decisions (diagnosis, quality control) where stakeholders need to understand why AI systems make specific pixel-level predictions about images.

Existing classification-based explanation methods require modification for segmentation tasks, meaning practitioners cannot simply apply off-the-shelf explainability tools and must carefully adapt techniques, which could lead to misapplication if not done thoughtfully.