Robustness of Visual Explanations to Common Data Augmentation

Visual explanations are less reliable than model predictions: The research found that post-hoc explainability methods—tools designed to show why AI models make decisions—are typically less robust to common image transformations than the models' actual classification performance, raising concerns about their trustworthiness for understanding AI behavior.

Different explanation methods have varying reliability: Some explainability techniques like LRP composites and Guided Backprop proved more stable when images were transformed, while others were less consistent, suggesting that the choice of explanation method significantly impacts how trustworthy the insights are.

Training data augmentation doesn't guarantee stable explanations: Even when models are trained with data augmentation (a common practice to improve robustness), their visual explanations remain unstable under transformations like rotation, scaling, and translation—meaning the explanations can change even when the model's predictions stay the same.

Explanations should respond predictably to image changes: The researchers expected explanations to either stay the same (invariant) for certain changes like color adjustments, or change in a predictable way (equivariant) for transformations like rotation. Finding inconsistencies suggests current explanation methods may not be reliably showing what the model actually "sees."

Current explainability methods need improvement for responsible AI: These findings highlight a gap between model performance and explanation reliability, suggesting that organizations relying on visual explanations to validate AI decisions should be cautious, as the explanations may not accurately reflect the model's reasoning.