The Inadequacy of Shapley Values for Explainability

Shapley values, a popular tool for explaining AI decisions, can produce fundamentally misleading results. The paper demonstrates that these methods can incorrectly rank irrelevant features as more important than features that actually drive predictions—the opposite of what users need to know.

The problem is not a limitation of current implementations but a mathematical inevitability. This isn't a bug that can be fixed with better algorithms; the paper shows this misleading behavior is built into how Shapley values work, meaning organizations relying on them for explanations may have false confidence in their understanding of their AI systems.

Finding truly reliable feature importance explanations may be computationally impossible for many real-world AI systems. The paper argues that creating rigorous, efficient explanation methods for complex classifiers faces fundamental computational barriers, suggesting there may be no easy solution to this problem.

Current explainability methods may give users false confidence in understanding AI decisions. If widely-used explanation techniques can systematically misrepresent which features matter, stakeholders making decisions based on these explanations could be acting on incorrect information without realizing it.

The responsible AI community needs to reconsider how it validates and deploys explanation tools. Organizations using Shapley values or similar methods for high-stakes decisions (hiring, lending, healthcare) should be aware these explanations may not reliably reflect what their AI systems actually learned.