Data Representing Ground-Truth Explanations to Evaluate XAI Methods

Current XAI evaluation methods lack objective standards: The paper identifies that existing approaches to evaluating explainability methods (such as comparing against other attribution approaches or using sensitivity analyses) don't measure whether explanations are actually accurate or indicate where methods fail, making it impossible to determine which XAI method is genuinely better than others.

Ground-truth benchmarks could transform XAI research: By proposing synthetic datasets with "canonical equations" representing ground-truth explanations, the authors suggest the field could adopt the same rigorous benchmarking approach used in other scientific fields, providing clear guidance on which research directions advance the field.

Subjectivity of explanations is addressable: While acknowledging that explanations can be subjective (satisfying different users differently), the paper demonstrates that this challenge can be overcome through structured methodology, suggesting a path forward for more objective evaluation despite inherent subjectivity.

Accountability requires accuracy measurement: The paper emphasizes that supporting accountable AI decisions requires measuring explanation accuracy—a capability currently missing from most XAI evaluation approaches, which is critical for responsible deployment of AI systems.

LIME and other popular methods need rigorous testing: The authors evaluate LIME (a widely-used XAI method) using their proposed datasets, suggesting that even established, popular explainability methods may have unidentified weaknesses that rigorous benchmarking could reveal.