Towards Robust and Accurate Stability Estimation of Local Surrogate Models in Text-based Explainable AI

Explanations from AI systems can be manipulated without changing the AI's actual decision, a vulnerability particularly concerning in high-stakes domains like medical diagnosis or legal proceedings where explanations directly influence human decision-making.

The tools used to measure whether explanations are reliable are themselves flawed, with many standard similarity measures being "overly sensitive" and producing inaccurate assessments of how vulnerable explanation methods actually are to manipulation.

Local surrogate methods like LIME, which are widely used to explain AI decisions, have documented weaknesses that remain poorly understood, highlighting a gap between knowing these methods are vulnerable and understanding why they fail.

Choosing the right measurement approach matters significantly for responsible AI deployment—using inappropriate similarity measures can lead organizations to incorrectly believe their explanation systems are more trustworthy than they actually are, potentially enabling unsafe AI applications.

The paper proposes incorporating linguistic knowledge (like word synonyms) into stability measurements to provide more accurate assessments of explanation reliability, suggesting that better evaluation methods can help identify and address real vulnerabilities in XAI systems.