AI2030 Responsible AI Tools Marketplace

Key Takeaways

Explanations from AI models aren't always trustworthy: Current methods that highlight which words influenced a text classification decision often fail to accurately reflect what the model actually used to make its prediction, raising concerns about relying on these explanations for high-stakes decisions.
The REGEX method makes explanations more faithful through two key improvements: The approach strengthens model robustness using input gradient regularization and adversarial training techniques, then aligns the model's attention mechanisms with feature attribution methods to ensure explanations genuinely reflect the decision-making process.
Improved explanations remain reliable even in unfamiliar contexts: Testing on out-of-domain datasets showed that REGEX's explanations maintained their fidelity across different data distributions, suggesting the method produces more genuinely faithful explanations rather than just appearing more convincing.
Humans can use these better explanations to build simpler, equally effective models: When researchers used the highlighted explanations from REGEX to train simpler "select-then-predict" models, they achieved comparable performance to complex end-to-end systems, demonstrating that faithful explanations enable more interpretable and potentially more trustworthy AI systems.
The improvement works across multiple explanation methods and datasets: Results were consistent across six different datasets and five different attribution methods, suggesting this approach to improving explanation faithfulness is broadly applicable rather than limited to specific scenarios.

Abstract

Feature attribution methods highlight the important input tokens as explanations to model predictions, which have been widely applied to deep neural networks towards trustworthy AI. However, recent works show that explanations provided by these methods face challenges of being faithful and robust. In this paper, we propose a method with Robustness improvement and Explanation Guided training towards more faithful EXplanations (REGEX) for text classification. First, we improve model robustness by input gradient regularization technique and virtual adversarial training. Secondly, we use salient ranking to mask noisy tokens and maximize the similarity between model attention and feature attribution, which can be seen as a self-training procedure without importing other external information. We conduct extensive experiments on six datasets with five attribution methods, and also evaluate the faithfulness in the out-of-domain setting. The results show that REGEX improves fidelity metrics of explanations in all settings and further achieves consistent gains based on two randomization tests. Moreover, we show that using highlight explanations produced by REGEX to train select-then-predict models results in comparable task performance to the end-to-end method.

Key Takeaways

Explanations from AI models aren't always trustworthy: Current methods that highlight which words influenced a text classification decision often fail to accurately reflect what the model actually used to make its prediction, raising concerns about relying on these explanations for high-stakes decisions.
The REGEX method makes explanations more faithful through two key improvements: The approach strengthens model robustness using input gradient regularization and adversarial training techniques, then aligns the model's attention mechanisms with feature attribution methods to ensure explanations genuinely reflect the decision-making process.
Improved explanations remain reliable even in unfamiliar contexts: Testing on out-of-domain datasets showed that REGEX's explanations maintained their fidelity across different data distributions, suggesting the method produces more genuinely faithful explanations rather than just appearing more convincing.
Humans can use these better explanations to build simpler, equally effective models: When researchers used the highlighted explanations from REGEX to train simpler "select-then-predict" models, they achieved comparable performance to complex end-to-end systems, demonstrating that faithful explanations enable more interpretable and potentially more trustworthy AI systems.
The improvement works across multiple explanation methods and datasets: Results were consistent across six different datasets and five different attribution methods, suggesting this approach to improving explanation faithfulness is broadly applicable rather than limited to specific scenarios.

Abstract

Towards Faithful Explanations for Text Classification with Robustness Improvement and Explanation Guided Training

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links

Towards Faithful Explanations for Text Classification with Robustness Improvement and Explanation Guided Training

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links