Don't Just Translate, Agitate: Using Large Language Models as Devil's Advocates for AI Explanations

Translating AI explanations into plain language doesn't automatically improve understanding: The paper challenges the common assumption that making technical explanations more readable through LLMs necessarily helps users understand AI systems better, suggesting that accessibility alone is insufficient for genuine comprehension.

Natural language explanations can create a false sense of transparency: When LLMs convert technical outputs into human-friendly summaries without acknowledging model limitations, uncertainties, or failure cases, they may actually reinforce user overconfidence in AI systems rather than fostering true transparency—a phenomenon the authors call the "illusion of interpretability."

LLMs should actively challenge AI explanations rather than simply repackage them: Instead of passively translating technical outputs, the paper proposes that LLMs should function as "devil's advocates" by surfacing alternative interpretations, potential biases, training data gaps, and scenarios where the model's reasoning breaks down.

Critical engagement with AI systems reduces harmful overreliance: By presenting counterarguments and limitations alongside explanations, this adversarial approach can help users develop a more skeptical, nuanced relationship with AI outputs and avoid misplaced trust in flawed or incomplete explanations.

Current XAI practices may inadvertently undermine responsible AI deployment: The paper warns that the trend of using LLMs to make explanations more palatable risks prioritizing user satisfaction over genuine accountability, potentially hindering efforts to build trustworthy AI systems.