ExplainabilityAccountabilityTrustworthinesscs.AI
Explaining Black-Box Algorithms Using Probabilistic Contrastive Counterfactuals
Sainyam Galhotra, Romila Pradhan, Babak Salimi
March 22, 2021
SIGMOD Conference
127 citations
11 influential
Key Takeaways
-
Causality matters more than correlation for AI explanations: The paper argues that existing explainability methods focus on associational relationships between inputs and outputs, but a causality-based approach using "counterfactual" reasoning (asking "what if" scenarios) provides more meaningful explanations of why an algorithm made a specific decision.
-
Explanations should include actionable recourse: Beyond just explaining a decision, LEWIS provides specific, concrete changes individuals can make to change an algorithm's outcome if they've been negatively affected—for example, telling someone exactly what factors they'd need to change to get approved for a loan rather than just explaining why they were denied.
-
One-size-fits-all explanations don't work: The framework is designed to generate explanations at multiple levels (local for individual cases, global for overall patterns, and contextual for specific situations) and can work with users who have different levels of technical knowledge, making AI decisions more accessible to non-experts.
-
Black-box algorithms can be explained without access to their internals: Unlike some methods requiring knowledge of how an algorithm works internally, LEWIS only needs input-output data, making it practical for real-world scenarios where companies won't share proprietary algorithm details.
-
Philosophical foundations improve AI trustworthiness: By grounding the approach in how humans naturally explain decisions (through counterfactual reasoning), the method aligns AI explanations with human cognition, potentially increasing genuine understanding and appropriate trust in algorithmic systems.