Reliable Explanations or Random Noise? A Reliability Metric for XAI

Popular explanation methods like SHAP and Integrated Gradients are unreliable in practice: Despite being theoretically well-motivated, these widely-used techniques produce substantially different explanations when inputs are slightly changed, features are represented differently, or models are updated—raising serious concerns about their trustworthiness in high-stakes domains like healthcare and finance.

The paper introduces a new measurement system (ERI) to test explanation stability: The Explanation Reliability Index provides the first systematic way to evaluate whether AI explanations remain consistent under four realistic conditions: small input changes, redundant features, model updates, and shifts in data distribution.

Explanation reliability has been largely ignored until now: The research reveals that while explaining AI decisions has become essential in critical sectors like energy systems and autonomous vehicles, no one has been systematically measuring whether these explanations are actually reliable—a significant gap in responsible AI practice.

Reliable explanations should pass four key tests: The paper establishes that trustworthy explanations must remain stable when inputs are slightly perturbed, consistent when features are represented differently, smooth as models evolve over time, and resilient when data patterns shift—providing a concrete framework for accountability.

A new benchmark (ERI-Bench) enables stress-testing of explanation methods: The authors created a testing framework to systematically evaluate explanation reliability across both synthetic and real-world datasets, providing practitioners with tools to assess whether their chosen explanation methods are dependable before deployment.