Right Predictions, Misleading Explanations: On the Vulnerability of Vision-Language Model Explanations

Explanations can be manipulated independently of predictions: The research demonstrates that vision-language models like CLIP can be attacked in ways that preserve correct predictions while completely changing the explanations (heatmaps) the model provides. This means a model could make the right decision for the wrong reasons without anyone noticing.

Current explanation methods may not reflect actual model reasoning: The study reveals a fundamental disconnect between what a model predicts and what its explanation mechanisms show. This undermines the transparency and trustworthiness that explanations are supposed to provide in high-stakes applications requiring human oversight.

Imperceptible perturbations can fool explanation systems: The X-Shift attack successfully redirects explanation heatmaps toward irrelevant image regions using changes so subtle that humans cannot detect them, while the model's predictions remain unchanged. This suggests explanation systems are vulnerable to manipulation that wouldn't be caught by standard robustness testing.

Standard adversarial defenses are insufficient: The paper shows that conventional adversarial attacks designed to prevent misclassification fail to address explanation manipulation, indicating that current safety measures don't protect against this specific vulnerability.

The problem generalizes across multiple models and explanation methods: The attack works across different CLIP architectures and various explanation techniques, suggesting this is a systemic issue rather than a problem limited to specific implementations, with implications for deploying these systems in real-world applications.