Unraveling the Dilemma of AI Errors: Exploring the Effectiveness of Human and Machine Explanations for Large Language Models

Human-created explanations outperform automated ones: When researchers compared different explanation methods, participants found explanations created by humans to be significantly more helpful than those generated by machines (including advanced methods like integrated gradients and ChatGPT), suggesting that human judgment remains valuable in making AI systems understandable.

Explanations can paradoxically harm decision-making: The study revealed a troubling trade-off: when explanations were perceived as helpful, participants actually performed worse on tasks, particularly when those explanations supported incorrect AI predictions. This suggests that clear explanations can increase trust in wrong answers rather than help people identify errors.

The "AI errors dilemma" requires new approaches: Current explanation methods face a fundamental challenge—they work well at building confidence in AI outputs, but this confidence can be misplaced when the AI is wrong. This finding indicates that existing explainability techniques may need redesign to help users recognize and question incorrect predictions rather than simply understand them.

Trust and performance move in opposite directions: The research found that higher trust in both the AI model and its explanations correlated with lower task performance, indicating that making AI systems more interpretable doesn't automatically make them safer or more reliable for decision-making.

Implications for real-world AI deployment: These findings suggest that organizations deploying AI systems should be cautious about assuming that better explanations automatically lead to better outcomes, and may need additional safeguards (like uncertainty indicators or contradiction detection) to prevent users from confidently acting on incorrect AI-generated answers.