Empirical Characterization of Rationale Stability Under Controlled Perturbations for Explainable Pattern Recognition

Explanation consistency matters for trustworthy AI: The paper demonstrates that even when AI models make correct predictions, their explanations (which features they say influenced their decision) can be inconsistent across similar inputs. This inconsistency is a red flag for responsible AI deployment, as it suggests models may be relying on unreliable reasoning patterns rather than genuinely understanding the task.

Current AI evaluation methods miss important problems: Most existing tests evaluate AI explanations one example at a time, missing systematic issues like biased feature reliance or inconsistent reasoning across similar cases. The paper's new metric catches these problems by comparing explanation patterns across multiple similar inputs, revealing hidden inconsistencies that standard tests overlook.

Small input changes can expose explanation instability: When researchers made minor, label-preserving changes to text inputs (like paraphrasing), they found that models' explanations sometimes shifted dramatically—even though the correct prediction remained the same. This suggests explanations may not reflect genuine model reasoning and could mislead users trying to understand why the AI made a decision.

The problem affects multiple AI models: Testing across different sentiment analysis models (BERT, RoBERTa, DistilBERT), the researchers found explanation inconsistency is a widespread issue, not limited to one architecture. This suggests the problem is systemic and should be considered when deploying any explanation-based AI system.

Better evaluation tools are needed before deployment: By proposing a metric to measure explanation consistency, this work provides a practical tool for organizations to audit whether their AI systems' explanations are reliable before putting them into real-world use.