Seeing Isn't Always Believing: Analysis of Grad-CAM Faithfulness and Localization Reliability in Lung Cancer CT Classification

Grad-CAM explanations can be misleading in medical AI: While this popular visualization technique effectively highlights tumor regions in traditional convolutional networks like ResNet and DenseNet, the study found that "Grad-CAM explanations may not always correspond to the true diagnostic evidence used by the networks," raising concerns about whether doctors can trust these visual explanations.

Different AI models produce conflicting explanations for the same diagnosis: The research revealed "substantial variability in saliency localization" across five different neural network architectures, meaning the same lung cancer image could produce different highlighted regions depending on which model made the prediction—a critical issue for clinical reliability.

Vision Transformers are particularly problematic for current explanation methods: Grad-CAM's "interpretive fidelity significantly degrades for Vision Transformer models due to non-local attention behavior," suggesting that newer AI architectures may be fundamentally incompatible with existing explanation techniques, creating a gap between model sophistication and explainability.

Current XAI methods need stronger validation before clinical use: The study introduces a quantitative framework combining "localization accuracy, perturbation-based faithfulness, and explanation consistency," highlighting that visual explanations alone are insufficient and require rigorous testing to verify they actually reflect how AI models make diagnostic decisions.

The field needs better safeguards for medical AI deployment: These findings expose "critical limitations of current saliency-based XAI approaches in medical imaging," suggesting that hospitals and regulators should be cautious about deploying AI systems whose decision-making cannot be reliably explained to clinicians.