Towards trustable SHAP scores

SHAP scores, a popular tool for explaining AI decisions, can sometimes give misleading information about which features actually matter most — recent research has identified cases where these explanations don't accurately reflect how a model makes decisions, which is problematic when organizations rely on them for high-stakes decisions.

The paper proposes modifications to how SHAP scores are calculated that fix known problems while maintaining the mathematical principles (Shapley values) that make them theoretically sound — these improvements avoid the misleading cases identified in prior work without requiring a complete overhaul of existing systems.

The proposed fixes can be implemented in existing SHAP software with minimal performance impact, making them practical for organizations already using these tools to explain their AI systems.

Not all AI models are equally difficult to explain with the improved SHAP approach — the paper identifies specific types of classifiers where computing trustworthy explanations is computationally feasible, which matters for real-world deployment in resource-constrained settings.

This work addresses a critical gap in responsible AI: ensuring that the explanations we give to stakeholders about how AI systems work are actually reliable, rather than appearing rigorous while potentially misleading decision-makers about what's driving algorithmic outcomes.