When Can You Trust Your Explanations? A Robustness Analysis on Feature Importances

Explanations can be unreliable even when AI systems perform well: The paper reveals that current explanation methods for AI models may give inconsistent results when data is slightly altered, meaning practitioners cannot always trust the explanations they receive—even if the model's predictions seem accurate.

Robustness of explanations should be a standard requirement: Just as we test whether AI systems work reliably, we need standardized tests to verify that the explanations provided are robust and trustworthy. The paper argues this validation step is currently missing from most AI development practices.

Small changes to input data can dramatically shift explanations: Using perturbations that stay within realistic data distributions, the researchers found that feature importance explanations can change significantly, suggesting that explanations may highlight unstable or unreliable patterns rather than true decision drivers.

Combining multiple explanations improves trustworthiness: The paper demonstrates that aggregating explanations from different methods produces more stable and reliable results, offering a practical technique for organizations seeking to validate their AI explanations before deployment.

This framework helps organizations meet regulatory requirements: With regulations increasingly demanding transparent and accountable AI systems, this robustness analysis approach provides practitioners with concrete tools to evaluate whether their explanations are trustworthy enough for real-world use.