Assessing Alignment and Stability of Feature Importance Explanations via Weight of Evidence

Feature importance explanations need rigorous testing, not just scores: The paper demonstrates that simply reporting attribution scores from popular explanation methods like LIME and SHAP is insufficient; these methods require systematic evaluation against reference standards to determine whether their explanations are actually reliable.

A new framework measures how well explanations match what we already know: The Weight of Evidence approach allows organizations to test whether AI explanations align with domain expert knowledge or ground truth, providing a concrete way to validate that explanation methods are giving trustworthy insights rather than misleading ones.

Explanations can be unstable even when they look reasonable: The research establishes a theoretical connection between Weight of Evidence and attribution variance, suggesting that some explanation methods may produce different results under slightly different conditions—a critical concern for deploying AI systems in high-stakes decisions.

This framework is practical for real-world AI auditing: By testing LIME and SHAP explanations across different scenarios, the paper shows the approach can work with existing, widely-used explanation tools, making it feasible for organizations to implement stronger quality checks on their AI systems.

Responsible AI requires comparing explanations to multiple reference points: The flexibility to test explanations against domain knowledge, ground truth, or other standards means organizations can choose validation approaches suited to their specific context and risk tolerance.