Do Not Trust Additive Explanations

Additive explanation methods like LIME and SHAP may be unreliable for complex models with feature interactions. The paper questions whether these popular "model agnostic" tools—widely promoted as solutions to AI transparency—actually provide faithful explanations when the underlying model relies on features that interact with each other rather than contributing independently.

Feature interactions are common enough to be a practical concern. The paper's large-scale benchmark found that additive explanations frequently produce misleading results, suggesting this isn't a rare edge case but a systematic problem affecting real-world applications of these explanation tools.

Current popular XAI methods may give false confidence in understanding AI decisions. Since LIME, SHAP, and similar tools are increasingly used to justify high-stakes decisions (hiring, lending, healthcare), their tendency to misrepresent non-additive models could lead organizations to trust explanations that don't accurately reflect how their AI systems actually work.

The paper introduces a method to detect when additive explanations are unreliable. Rather than abandoning these tools entirely, the authors provide a way to identify instances where feature interactions make additive explanations untrustworthy, allowing practitioners to know when they should be skeptical of the results.

Explainability alone doesn't guarantee trustworthiness. This work demonstrates that having an explanation doesn't mean that explanation is correct—a critical distinction for responsible AI deployment where decisions based on faulty explanations could harm individuals or perpetuate biases.