From unbiased MDI Feature Importance to Explainable AI for Trees

Tree-based models need better transparency tools: The paper identifies that popular tree models like Random Forests use a default importance measure (Gini importance) that is biased, meaning it can misleadingly overstate how important certain features are in making predictions—a critical issue for understanding AI decisions.

Out-of-bag methods offer a solution: The research demonstrates that "out-of-bag based bias correction methods" provide a more reliable way to measure feature importance by using data the model hasn't seen during training, helping practitioners get more accurate explanations of what their models actually learned.

Explainable AI algorithms for trees have a hidden flaw: The paper points out that newly developed explainable AI tools for tree-based models introduce their own bias by including "inbag data" (data used to train the model), which can distort local explanations of individual predictions.

Unifying framework matters for responsible AI: By connecting different bias correction approaches and explanation methods, this work provides a clearer foundation for practitioners to choose appropriate tools, reducing the risk of making decisions based on misleading model explanations.

Feature importance transparency is essential for trust: Accurate measurement of which features drive model decisions is fundamental to responsible AI deployment, particularly in high-stakes domains where stakeholders need to understand and validate why an AI system made a specific decision.