XAI-TRIS: Non-linear image benchmarks to quantify false positive post-hoc attribution of feature importance

Popular explanation methods often fail to identify truly important features: The study found that widely-used XAI methods frequently cannot outperform random guessing when identifying which features actually matter for a model's decisions, raising serious questions about their reliability in real-world applications.

Explanations vary dramatically depending on the model architecture used: Even when different neural network architectures solve the same problem, they produce vastly different explanations for their decisions. This means the same AI system could justify its answer in contradictory ways depending on its underlying design, creating a risk of misinterpretation.

Current XAI methods lack rigorous theoretical foundations: The paper highlights that explanation methods have been widely adopted without formal verification of what their outputs actually mean, making it unclear what conclusions users can safely draw from these explanations in high-stakes decisions.

Controlled testing reveals fundamental limitations: By creating benchmark datasets where the correct explanations were known in advance, researchers demonstrated that popular XAI methods struggle with non-linear problems—the exact type of complex decision-making that deep neural networks typically handle.

The field needs better validation standards: This work emphasizes that XAI methods require the same rigorous testing and benchmarking that other AI components receive, rather than being deployed based on citations and intuitive appeal alone.