ExplainabilityAccountabilityTransparencystat.ML
Interpretable Model-Aware Counterfactual Explanations for Random Forest
Joshua S. Harvey, Guanchao Feng, Sai Anusha Meesala, Tina Zhao, Dhagash Mehta
October 31, 2025
arXiv.org
Key Takeaways
-
Counterfactual explanations are more actionable than traditional methods: The paper argues that telling someone "to get approved for a loan, you would need to increase your savings by $5,000" (counterfactual) is more useful and intuitive than complex statistical attribution scores, particularly for regulated industries like finance where decisions must be justified to customers.
-
The method produces simpler, more interpretable explanations: When tested on credit decisions, the approach generated "sparser" explanations than Shapley values—a popular AI explanation technique—meaning it identified fewer critical factors, making it easier for non-experts to understand why an AI system made a specific decision.
-
AI explanations should reflect how models actually work: Rather than applying generic explanation frameworks to any model, this research shows that explanations are more reliable when they're tailored to how the specific model (in this case, random forests) actually makes decisions internally, improving alignment between the explanation and the model's reasoning.
-
Regulatory compliance requires causal, not just correlational explanations: The paper highlights that regulated industries need explanations showing what would need to change to alter an outcome, not just which factors mattered most—a distinction critical for fair lending, hiring, and other high-stakes decisions where people have a right to understand how to challenge adverse decisions.