A Turing Test for Transparency

Explanations can backfire: Research shows that when AI systems provide explanations for their predictions, humans often trust those predictions more, even when the predictions are actually wrong—the opposite of what explainable AI is supposed to achieve.

The "human-like" trap: This increased trust appears to occur because explanations that seem intuitive or human-like are more persuasive to people, regardless of whether the underlying AI prediction is correct. This suggests that making AI explanations too natural-sounding may actually undermine responsible AI use.

A new test for transparency: The authors propose a "Turing Test for Transparency" where human evaluators try to distinguish between explanations written by humans versus generated by AI systems. If humans cannot reliably tell the difference, the explanation may be dangerously persuasive and mask potential errors.

Trust calibration is critical: For responsible AI deployment, organizations need to ensure humans can actually detect when explanations come from machines rather than people—this ability is essential for maintaining appropriate, calibrated trust in AI systems rather than blind reliance.

Current XAI methods may pose risks: Early experimental results on text classification tasks suggest that even basic machine learning models can generate explanations that fool humans, indicating that many current transparency methods may inadvertently increase inappropriate trust in AI predictions.