XAI-Driven Diagnosis of Generalization Failure in State-Space Cerebrovascular Segmentation Models: A Case Study on Domain Shift Between RSNA and TopCoW Datasets

Domain shift is a critical barrier to AI deployment in medicine: The study demonstrates a dramatic real-world example where a high-performing cerebrovascular segmentation model (Dice score of 0.86) failed catastrophically on a different dataset (Dice score of 0.29), highlighting why models trained on one medical imaging dataset cannot simply be assumed to work on others.

Standard performance metrics alone are insufficient for trustworthy AI: The paper shows that relying only on accuracy scores masks the true problem—the model was confidently making wrong predictions rather than admitting uncertainty, which is particularly dangerous in clinical settings where false confidence could lead to misdiagnosis.

Explainable AI (XAI) tools can diagnose why models fail, not just that they fail: By using visualization techniques (Seg-XRes-CAM) to examine the model's attention mechanisms, researchers discovered the specific cause of failure: the model had learned to focus on the wrong visual features when encountering new data, rather than abandoning anatomical structures entirely.

Technical differences between datasets have real consequences: Specific variations in image resolution and background noise between the two datasets caused the model to lose track of actual blood vessels, demonstrating that seemingly minor technical differences in how medical images are acquired can undermine AI reliability.

Moving toward responsible medical AI requires deeper investigation beyond performance numbers: This work exemplifies how combining explainability methods with rigorous analysis is necessary to build trustworthy AI systems that clinicians can safely deploy across different hospitals and imaging equipment.