From SHAP Scores to Feature Importance Scores

SHAP and LIME, despite their widespread use, can produce misleading feature importance rankings — the paper demonstrates that the exact computation of feature attributions using these popular tools can yield "manifestly unsatisfactory results," raising concerns about their reliability for explaining AI decisions in high-stakes applications.

Feature importance scoring is fundamentally connected to voting power theory — the research reveals that rigorous approaches to feature attribution are mathematically equivalent to power indices from voting theory, suggesting that decades of research on measuring influence in voting systems can be applied to improve AI explainability.

Current feature importance methods lack agreed-upon standards for what makes a "good" explanation — the paper proposes novel desirable properties that feature importance scores should exhibit, addressing a gap where it remains unclear which of the many available methods best serve the goal of providing trustworthy feature attributions without producing unsatisfactory or misleading results.

Multiple valid approaches exist for feature importance, but their suitability for XAI varies — rather than identifying a single "best" method, the research suggests that different power indices and feature importance scores have different strengths, implying that practitioners need clearer guidance on which approach fits their specific explainability needs.