Ensuring Medical AI Safety: Interpretability-Driven Detection and Mitigation of Spurious Model Behavior and Associated Data

Shortcut learning poses serious risks in medical AI: Deep neural networks used in medical applications can learn spurious correlations (shortcuts) rather than genuine medical patterns, with potentially fatal consequences in clinical practice.

A comprehensive framework is more effective than isolated fixes: The Reveal2Revise approach combines both detection and mitigation of biased AI behavior in a single framework, rather than addressing these problems separately, leading to more robust medical AI systems.

Semi-automated tools can reduce expert burden: The enhanced framework uses interpretability-based methods to semi-automate the identification of biases at both the sample and feature levels, reducing the substantial labeling effort typically required from medical domain experts.

The approach works across different AI architectures and real-world scenarios: The framework successfully identified and corrected biased behavior in multiple model types (VGG16, ResNet50, Vision Transformers) and across four medical datasets with both controlled and real-world data artifacts, demonstrating broad applicability.

Improved interpretability directly improves safety: By making AI decision-making more transparent and understandable, the framework enables better detection of when models are relying on irrelevant data artifacts rather than actual medical indicators, ultimately increasing the trustworthiness of medical AI systems.