Why is plausibility surprisingly problematic as an XAI criterion?

Plausibility is an unreliable measure of true explainability: The paper argues that current metrics assessing how "convincing" an AI explanation seems to humans (based on feature localization or correlation) don't actually measure whether an explanation is valid or accurate, because they ignore the underlying assumptions required for explanations to be meaningful.

Plausible-sounding explanations can manipulate users: By optimizing for explanations that simply sound convincing rather than being truthful, AI systems may generate misleading explanations that give users false confidence in the AI's reasoning, ultimately undermining rather than enhancing responsible AI deployment.

Relying on plausibility damages user trust and autonomy: When users discover that plausible-sounding explanations were misleading, their trust in the AI system deteriorates. Additionally, prioritizing explanations that are merely convincing rather than accurate can manipulate users into accepting AI decisions without genuine understanding, compromising their ability to make autonomous choices.

Human explanations shouldn't be the gold standard for AI explanations: The paper challenges the assumption that human-style explanations are the best model for AI explainability, suggesting this approach limits exploration of alternative methods that might better serve transparency and understanding in AI systems.

The field should abandon plausibility as an evaluation criterion: Given these problems, the authors argue the XAI community should stop using plausibility metrics to evaluate and improve XAI algorithms, and instead pursue alternative approaches to genuinely enhance understandability and responsible AI.