Towards Rigorous Explainability by Feature Attribution

Current explanation methods can mislead decision-makers: The paper argues that popular non-symbolic explanation methods used today—including widely-adopted tools like SHAP—lack mathematical rigor and can provide misleading explanations of how AI models make decisions, which is particularly dangerous in high-stakes applications like healthcare or criminal justice.

Shapley values have fundamental limitations: Despite being adopted across the industry as a standard for explaining AI decisions, the paper identifies specific problems with how Shapley values are applied in explainable AI, suggesting they don't provide the rigorous guarantees users assume they do.

Symbolic methods offer a more trustworthy alternative: The paper advocates for "rigorous symbolic methods" as a replacement for current explanation approaches, implying that mathematical approaches with formal guarantees could provide more reliable insights into why AI systems make specific decisions.

High-stakes AI decisions demand higher standards: The paper emphasizes that the lack of rigor in current explanation methods is especially problematic when AI influences important decisions affecting people's lives, suggesting that different standards should apply depending on how consequential the AI application is.

The field needs to move toward provable explanations: Rather than accepting explanations that seem reasonable, the paper calls for the XAI field to prioritize methods that can be mathematically proven to accurately represent how models work, similar to standards in other rigorous scientific fields.