A Survey on the Robustness of Feature Importance and Counterfactual Explanations

Explanations of AI models may not be reliable: The paper reveals that popular methods used to explain how AI models make decisions—specifically feature importance and counterfactual explanations—often lack robustness, meaning small changes to the data or model can produce drastically different explanations, even when the model's predictions remain the same.

Current explanation methods need validation before deployment: Since these explanation techniques are widely used in high-stakes domains like finance, the survey emphasizes that organizations should not assume explanations accurately reflect how their models actually work without first testing their robustness and reliability.

A unified framework for testing explanations is needed: The paper proposes a taxonomy to classify different approaches for measuring robustness, suggesting that the field needs standardized methods to evaluate whether explanations can be trusted—similar to how we test model accuracy.

Small perturbations can reveal explanation fragility: The research highlights that explanations can change significantly with minor adjustments to input data or model parameters, raising concerns about whether these explanations truly capture the model's underlying decision-making logic or are merely artifacts of the explanation method itself.

Responsible AI requires reliable explainability: For AI systems to be trustworthy and accountable—particularly in regulated industries like finance—organizations must move beyond simply having explanations to ensuring those explanations are robust and accurately represent how their models behave.