ExplainabilityTransparencyAccountabilitycs.LG
Towards Attributions of Input Variables in a Coalition
Xinhao Zheng, Huiqi Deng, Quanshi Zhang
September 23, 2023
International Conference on Machine Learning
1 citations
Key Takeaways
-
Attribution conflicts are a real problem in AI explanations: The paper identifies that current methods produce contradictory explanations when attributing credit to groups of input variables versus individual ones—for example, two features might each get credited with 10% importance, but together they're credited with only 15%, creating confusion about how AI models actually work.
-
How variables interact matters for fair credit assignment: The research shows that specific types of interactions between input variables (AND-OR operations) are responsible for these conflicts, suggesting that better AI explanations require understanding not just individual features but how they work together in the model's decision-making process.
-
New metrics can measure whether explanations make sense: The authors propose three new metrics to evaluate whether variable groupings produce faithful explanations, providing a practical tool for AI developers and auditors to verify that their explanation methods are actually reliable and consistent.
-
The approach works across different AI applications: Validation on diverse tasks—from natural language processing to image recognition to game-playing AI—suggests these findings have broad relevance for improving explanations across many types of AI systems, not just specialized cases.
-
Theoretical guidance is needed for building explanations: Rather than letting developers arbitrarily decide which variables to group together when explaining AI decisions, this work provides principled, mathematically-grounded guidance for forming meaningful variable partitions that lead to more trustworthy explanations.