Does Explainability Transfer? A Controlled Benchmark of Attribution Methods on Vision Transformers and CNNs

Explainability methods don't work the same across different AI architectures: The paper's core finding is that attribution methods ranked as "best" for traditional CNNs often perform poorly on Vision Transformers, meaning conclusions about XAI effectiveness established on older models cannot be assumed to apply to newer transformer-based systems now widely used in computer vision.

Popular explanation methods may appear better than they actually are: CAM-based methods scored highest on standard localization tests, but the paper found this largely reflects "metric saturation"—essentially, the evaluation metrics themselves were too easy—rather than genuinely accurate explanations of where the AI is looking.

There's a trade-off between explanation quality and stability: Attention rollout provided the most consistent and robust explanations across different model types, but these explanations had poor localization accuracy, suggesting organizations must choose between stable but less precise explanations versus precise but fragile ones.

Current evaluation methods have significant limitations: The paper found that "faithfulness correlation offers limited discrimination between attribution methods," meaning the standard ways we measure whether explanations are trustworthy may not effectively distinguish between good and poor explanation methods.

Architecture matters more than method choice: The study evaluated 13 different explanation methods across 8 model types, revealing that which architecture the AI uses is a stronger predictor of explanation quality than which explanation technique is applied.