ExplainabilityTransparencyAccountabilitycs.LG
Rational Shapley Values
David S. Watson
June 18, 2021
Conference on Fairness, Accountability and Transparency
30 citations
Key Takeaways
-
Shapley values alone aren't enough for real-world explanations: Traditional feature attribution methods like standard Shapley values treat all contexts the same way, but the paper demonstrates that effective AI explanations need to adapt to specific situations—what matters for explaining a healthcare decision differs from what matters in finance.
-
The method combines two previously incompatible explanation approaches: Rational Shapley values bridge feature attributions (which are easy to understand but lack context) and counterfactuals (which are realistic but hard to summarize), offering a more complete picture of why an AI system made a particular decision.
-
User knowledge and goals should shape explanations: Rather than providing one-size-fits-all explanations, this approach allows the explanation method to incorporate what users already know and what they specifically need to understand, making explanations more relevant and actionable for decision-makers in high-stakes domains.
-
Decision theory and causal reasoning improve explanation quality: By grounding the method in formal decision theory and causal modeling, the paper shows that explanations become more rigorous and reliable—important for building justified trust in AI systems used in healthcare, finance, and other critical applications.
-
Empirical validation shows practical improvements: The method outperforms existing XAI tools in both quantitative metrics and qualitative assessments, suggesting it could provide more trustworthy explanations for the complex models increasingly used to assist human decision-making.