Revealing Vulnerabilities of Neural Networks in Parameter Learning and Defense Against Explanation-Aware Backdoors

Explainability tools can be manipulated to mislead users: The paper demonstrates that AI explanation methods—which are designed to build trust—can be attacked in ways that make the AI appear to make decisions for the right reasons when it actually doesn't. These "blinding attacks" add invisible changes to images that fool both the AI's predictions and its explanations simultaneously.

Current AI explanations may hide security vulnerabilities: Even when an AI model maintains high accuracy, attackers can alter what the explanation shows without users noticing. This means relying solely on explanation methods to verify AI trustworthiness is insufficient, as the explanations themselves can be compromised.

Defense mechanisms can significantly reduce explanation attacks: The researchers developed a detection method that reduces the success rate of explanation-aware attacks by approximately 99% without requiring retraining of the AI model, suggesting practical solutions are feasible for protecting explanation systems during deployment.

Statistical analysis of AI internals reveals attacks: By examining changes in the neural network's internal parameters (weights), researchers can identify when an AI system has been compromised by these attacks, providing a technical foundation for building more robust safeguards.

Responsible AI requires protecting both predictions and explanations: This work highlights that trustworthy AI systems need security measures that go beyond just making accurate predictions—the explanations themselves must be protected from manipulation to maintain genuine transparency.