ExplainabilityTransparencyAccountabilitycs.CV
IG2: Integrated Gradient on Iterative Gradient Path for Feature Attribution
Yue Zhuo, Zhiqiang Ge
June 16, 2024
IEEE Transactions on Pattern Analysis and Machine Intelligence
36 citations
3 influential
Key Takeaways
-
Feature attribution methods need to account for both the model's behavior at the input being explained AND at the baseline comparison point. The paper demonstrates that previous Integrated Gradients approaches focused primarily on gradients from the input being explained, but neglecting the counterfactual baseline's gradients led to less accurate explanations. IG2 addresses this by incorporating both, potentially making AI explanations more reliable for auditing and understanding model decisions.
-
Arbitrary baseline selection has been a persistent problem in explainability methods. The paper identifies that choosing a baseline (the reference point for comparison) was previously done arbitrarily, which could lead to inconsistent or misleading feature importance scores. IG2's novel approach to baseline selection (GradCF) helps reduce this arbitrariness, making explanations more principled and reproducible across different use cases.
-
The proposed method reduces "attribution noise" in feature importance scores. By iteratively incorporating gradient information along the path between input and baseline, IG2 produces cleaner, more stable explanations. This is important for practitioners who need to trust that small changes in input won't cause wildly different explanations of why an AI system made a particular decision.
-
Improved explainability methods have been validated across diverse real-world applications. The research tested IG2 on multiple domains including image recognition (ImageNet, MNIST, CelebA), text analysis (TREC question answering), and industrial applications (wafer-map failure detection), demonstrating that better attribution methods can benefit various sectors relying on AI.