What Do You See? Evaluation of Explainable Artificial Intelligence (XAI) Interpretability through Neural Backdoors

Current XAI methods have significant blind spots: The research found that six out of seven tested explanation methods failed to completely highlight trigger regions in neural networks, meaning they missed important features that directly caused the AI's decisions—raising concerns about whether these popular explanation tools are actually showing us what matters.

Backdoor triggers provide a measurable way to test XAI reliability: Rather than relying on subjective human judgment, the researchers used hidden malicious patterns intentionally inserted into AI models as "ground truth" to objectively evaluate whether explanation methods could identify truly important features, offering a more rigorous testing approach for the field.

Most explanation methods struggle with local, feature-based approaches: The study specifically found that methods focusing on local explanations and feature relevance were particularly prone to missing critical regions, suggesting that certain popular XAI techniques may give users false confidence in their understanding of AI decisions.

Only one tested method fully revealed hidden triggers: A model-free explanation approach was the only one capable of uncovering entire trigger regions, indicating that the choice of XAI method significantly impacts whether users see the complete picture of what an AI system is actually using to make decisions.

Evaluation of XAI itself needs improvement: The paper highlights that we currently lack reliable ways to verify whether explanation methods are actually trustworthy, which is critical since these tools are increasingly used to build confidence in high-stakes AI applications like healthcare and criminal justice.