ExplainabilityTransparencyAccountabilitycs.AI
On Formal Feature Attribution and Its Approximation
Jinqiang Yu, Alexey Ignatiev, Peter J. Stuckey
July 7, 2023
arXiv.org
13 citations
Key Takeaways
-
Current explanation methods have serious flaws: Popular AI explanation techniques like LIME and SHAP can produce unsound explanations and rely on unrealistic data samples, meaning the explanations they provide may not accurately reflect how the AI model actually works.
-
Formal approaches offer soundness but face practical limits: While formal methods for explaining AI decisions are mathematically rigorous and avoid the pitfalls of LIME and SHAP, they struggle with scalability and cannot currently handle feature attribution—the task of identifying which input features most influence a model's decisions.
-
Formal feature attribution (FFA) provides a new solution: The paper proposes a new method that combines formal verification techniques with feature attribution, offering explanations that are both mathematically sound and capable of identifying which features matter most for specific predictions.
-
Explanations need to be concise to be useful: Even when AI explanations are formally correct, they are often too large and complex to be practically useful to decision-makers, highlighting the need for methods that produce shorter, more interpretable explanations alongside their formal guarantees.
-
Bridging the gap between rigor and practicality matters for responsible AI: This work addresses a key challenge in responsible AI—moving beyond choosing between mathematically sound but impractical explanations versus practical but potentially unreliable ones.
Abstract
Recent years have witnessed the widespread use of artificial intelligence (AI) algorithms and machine learning (ML) models. Despite their tremendous success, a number of vital problems like ML model brittleness, their fairness, and the lack of interpretability warrant the need for the active developments in explainable artificial intelligence (XAI) and formal ML model verification. The two major lines of work in XAI include feature selection methods, e.g. Anchors, and feature attribution techniques, e.g. LIME and SHAP. Despite their promise, most of the existing feature selection and attribution approaches are susceptible to a range of critical issues, including explanation unsoundness and out-of-distribution sampling. A recent formal approach to XAI (FXAI) although serving as an alternative to the above and free of these issues suffers from a few other limitations. For instance and besides the scalability limitation, the formal approach is unable to tackle the feature attribution problem. Additionally, a formal explanation despite being formally sound is typically quite large, which hampers its applicability in practical settings. Motivated by the above, this paper proposes a way to apply the apparatus of formal XAI to the case of feature attribution based on formal explanation enumeration. Formal feature attribution (FFA) is argued to be advantageous over the existing methods, both formal and non-formal. Given the practical complexity of the problem, the paper then proposes an efficient technique for approximating exact FFA. Finally, it offers experimental evidence of the effectiveness of the proposed approximate FFA in comparison to the existing feature attribution algorithms not only in terms of feature importance and but also in terms of their relative order.
Paper Info
arXiv ID2307.03380
PublishedJuly 7, 2023
RevisedAugust 28, 2023
Citations
13
Relevance82%
VenuearXiv.org
TypeJournalArticle
Categorycs.AI
Responsible AI Topics
ExplainabilityTransparencyAccountability
Links
Citation data from Semantic Scholar