ExplainabilityTransparencyAccountabilitycs.LG
Explaining black box decisions by Shapley cohort refinement
Masayoshi Mase, Art B. Owen, Benjamin Seiler
November 1, 2019
arXiv.org
63 citations
7 influential
Key Takeaways
-
Uses real data combinations instead of unrealistic scenarios: Unlike some explanation methods that artificially change input values in ways that might never occur in practice, this approach only examines combinations of people and variables that actually exist in the data, making explanations more grounded in reality.
-
Applies game theory principles to AI transparency: The method adapts Shapley values from cooperative game theory to measure how much each input variable contributes to a model's decision, providing a mathematically principled way to assign "credit" to different factors.
-
Focuses on similar groups rather than isolated variables: Instead of changing one variable while keeping others fixed (which can create logically impossible scenarios), the approach identifies groups of similar people on each variable and uses those natural groupings to understand what drives predictions.
-
Bridges two important fields in responsible AI: The work connects explainable AI research with global sensitivity analysis, suggesting these communities can learn from each other's approaches to understanding complex systems and improving model accountability.
-
Provides mathematically consistent explanations: The introduction of a "squared cohort Shapley value" ensures the method follows established mathematical principles (Shapley axioms), which helps guarantee the explanations are fair and logically consistent across different applications.