How Reliable and Stable are Explanations of XAI Methods?

XAI methods are vulnerable to small data changes: The study found that current explainability methods are "sensitive to perturbations," meaning they can produce different explanations when the input data is slightly altered. This raises concerns about whether we can trust these explanations in real-world applications where data naturally varies.

One method showed greater stability: Among the five XAI methods tested (Dalex, Eli5, eXirt, Lofo, and Shap), one method demonstrated resilience to perturbations, suggesting that not all explanation tools are equally unreliable—some may be more trustworthy for critical applications.

eXirt can identify unreliable models: The research found that the eXirt method was able to identify which machine learning models were most reliable, offering a potential safeguard for practitioners who need to assess whether a model's predictions can be trusted before deployment.

Explanation instability matters for real-world decisions: Since black box AI models are "increasingly being used in the daily lives of human beings," unstable explanations could mislead users, regulators, or decision-makers who rely on these tools to understand why an AI system made a particular decision (such as in healthcare or lending).

More rigorous testing of XAI tools is needed: This research highlights that explainability methods themselves require validation and stress-testing before being deployed as trustworthiness tools, rather than being assumed reliable simply because they exist.