ExplainabilityTransparencyAccountabilitycs.LG
Towards a Unified Evaluation of Explanation Methods without Ground Truth
Hao Zhang, Jiayi Chen, Haotian Xue, Quanshi Zhang
November 20, 2019
arXiv.org
12 citations
2 influential
Key Takeaways
-
Ground truth for AI explanations is often unavailable: The paper identifies a fundamental challenge in responsible AI—we typically cannot verify whether an explanation method is actually showing us what a neural network is "thinking," making it difficult to trust these explanations in high-stakes applications.
-
New evaluation metrics enable fairer comparison: The researchers developed four metrics that can assess explanation methods without needing ground-truth data, allowing for more objective evaluation of the nine different explanation techniques tested and helping practitioners choose more reliable methods.
-
Explanation methods vary significantly in quality: By applying these metrics across multiple popular explanation approaches, the paper reveals that different methods produce substantially different results, suggesting that the choice of explanation method matters considerably for understanding AI system decisions.
-
Standardized evaluation supports responsible AI deployment: The proposed framework moves toward establishing common standards for evaluating explanations, which is essential for building trust in AI systems and ensuring they can be properly audited before deployment in sensitive domains like healthcare or criminal justice.
Abstract
This paper proposes a set of criteria to evaluate the objectiveness of explanation methods of neural networks, which is crucial for the development of explainable AI, but it also presents significant challenges. The core challenge is that people usually cannot obtain ground-truth explanations of the neural network. To this end, we design four metrics to evaluate explanation results without ground-truth explanations. Our metrics can be broadly applied to nine benchmark methods of interpreting neural networks, which provides new insights of explanation methods.