ExplainabilityTransparencyTrustworthinesscs.LG
Revisiting the robustness of post-hoc interpretability methods
Jiawen Wei, Hugues Turbé, Gianmarco Mengaldo
July 29, 2024
arXiv.org
12 citations
Key Takeaways
-
Post-hoc interpretability methods—tools designed to explain AI decisions by identifying important data features—often disagree with each other, raising serious questions about whether any of them are actually reliable guides to how AI models make decisions.
-
Current evaluation methods only measure average performance across many samples, missing critical information about whether an explanation method works consistently for individual decisions; the paper introduces new "fine-grained" metrics to assess robustness on a case-by-case basis.
-
Robustness of explanations (consistency across similar cases) is generally linked to overall performance, suggesting that methods producing more reliable average results also tend to provide more dependable explanations for specific decisions—an important finding for practitioners choosing which explanation tool to trust.
-
The lack of sample-level assessment has been a blind spot in XAI evaluation, potentially allowing organizations to deploy explanation methods that work well on average but fail unpredictably for certain types of decisions, which could undermine trust in AI systems.
-
Better evaluation metrics for interpretability methods are essential for responsible AI deployment, as stakeholders need confidence that explanations are not just statistically sound on average but genuinely trustworthy when explaining individual, high-stakes decisions.
Abstract
Post-hoc interpretability methods play a critical role in explainable artificial intelligence (XAI), as they pinpoint portions of data that a trained deep learning model deemed important to make a decision. However, different post-hoc interpretability methods often provide different results, casting doubts on their accuracy. For this reason, several evaluation strategies have been proposed to understand the accuracy of post-hoc interpretability. Many of these evaluation strategies provide a coarse-grained assessment -- i.e., they evaluate how the performance of the model degrades on average by corrupting different data points across multiple samples. While these strategies are effective in selecting the post-hoc interpretability method that is most reliable on average, they fail to provide a sample-level, also referred to as fine-grained, assessment. In other words, they do not measure the robustness of post-hoc interpretability methods. We propose an approach and two new metrics to provide a fine-grained assessment of post-hoc interpretability methods. We show that the robustness is generally linked to its coarse-grained performance.
Paper Info
arXiv ID2407.19683
PublishedJuly 29, 2024
Citations
12
Relevance78%
VenuearXiv.org
TypeJournalArticle
Categorycs.LG
Responsible AI Topics
ExplainabilityTransparencyTrustworthiness
Links
Citation data from Semantic Scholar