AI2030 Responsible AI Tools Marketplace

Key Takeaways

Local explanations can be misleading at scale: The paper demonstrates that XAI methods producing plausible explanations for individual predictions may systematically misrepresent how AI models actually work overall. In medical contexts, this could mean a model appears to use clinically relevant patterns when it actually relies on irrelevant signal characteristics.
Standard computer vision methods fail in medical domains: Thirteen gradient-based explanation methods, many adapted from computer vision, were tested on ECG classification. Nine of the 13 methods performed below chance level for at least one heart condition, revealing that techniques successful in other fields cannot be automatically trusted in healthcare without domain-specific validation.
Explanations often follow signal amplitude rather than clinical relevance: The methods systematically prioritized high-amplitude regions of ECG signals over diagnostically important low-amplitude areas. For ischemia detection, one method (LRP-ε) assigned only 4.6% importance to the clinically critical ST segment, while an alternative method (LRP-SIGN) correctly assigned 63.8%—demonstrating how different approaches can produce drastically different explanations.
Clinical guidelines should ground AI evaluation: The paper introduces a framework that evaluates AI explanations against established medical knowledge about which signal regions matter for diagnosis. This "guideline-grounded" approach revealed failures that standard evaluation methods would have missed, suggesting responsible AI deployment in medicine requires domain expert involvement.
Global evaluation catches problems local inspection misses: Aggregating explanations across multiple heartbeats and comparing them systematically to clinical standards uncovered systematic biases that examining individual predictions would not reveal.

Abstract

Explainable AI (XAI) is used to assess whether artificial intelligence models rely on meaningful patterns, yet explanations that appear plausible for individual predictions may systematically misrepresent model behavior. This is particularly problematic in medicine, where models may rely on irrelevant signal characteristics rather than disease-specific patterns without being recognizable. We address this challenge using electrocardiogram (ECG) data, for which clinical guidelines provide explicit knowledge about diagnostically relevant signal regions. We introduce a global, guideline-grounded framework that aggregates explanations across heartbeats to evaluate them against clinically defined regions of interest. Using four binary classifiers trained on PTB-XL, we assess 13 gradient-based methods across two categories of patterns: low-amplitude segments and high-amplitude QRS morphology. Our results reveal a systematic failure of methods transferred from computer vision. Their explanations often follow signal amplitude rather than clinical relevance, with mean Spearman correlations up to 0.69, leading them to overlook diagnostically decisive low-amplitude regions. For ischemia, LRP-$ε$ assigns only 4.6% of relevance to the ST segment, compared with 63.8% for LRP-SIGN. Nine of 13 methods fall below chance for at least one condition, indicating inconsistent reliability across patterns. These findings show that global, domain-grounded evaluation can uncover systematic explanation failures not obvious from sample-level heatmaps.

Key Takeaways

Local explanations can be misleading at scale: The paper demonstrates that XAI methods producing plausible explanations for individual predictions may systematically misrepresent how AI models actually work overall. In medical contexts, this could mean a model appears to use clinically relevant patterns when it actually relies on irrelevant signal characteristics.
Standard computer vision methods fail in medical domains: Thirteen gradient-based explanation methods, many adapted from computer vision, were tested on ECG classification. Nine of the 13 methods performed below chance level for at least one heart condition, revealing that techniques successful in other fields cannot be automatically trusted in healthcare without domain-specific validation.
Explanations often follow signal amplitude rather than clinical relevance: The methods systematically prioritized high-amplitude regions of ECG signals over diagnostically important low-amplitude areas. For ischemia detection, one method (LRP-ε) assigned only 4.6% importance to the clinically critical ST segment, while an alternative method (LRP-SIGN) correctly assigned 63.8%—demonstrating how different approaches can produce drastically different explanations.
Clinical guidelines should ground AI evaluation: The paper introduces a framework that evaluates AI explanations against established medical knowledge about which signal regions matter for diagnosis. This "guideline-grounded" approach revealed failures that standard evaluation methods would have missed, suggesting responsible AI deployment in medicine requires domain expert involvement.
Global evaluation catches problems local inspection misses: Aggregating explanations across multiple heartbeats and comparing them systematically to clinical standards uncovered systematic biases that examining individual predictions would not reveal.

Abstract

Beyond Local Inspection: Global, Guideline-Grounded Evaluation of Post-hoc XAI Methods for ECG Classification

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links

Beyond Local Inspection: Global, Guideline-Grounded Evaluation of Post-hoc XAI Methods for ECG Classification

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links