Don't be Fooled: The Misinformation Effect of Explanations in Human-AI Collaboration

Incorrect explanations can be more harmful than no explanation at all: When AI systems provide the right answer but with wrong reasoning, humans internalize the flawed logic rather than learning correct procedures. This "misinformation effect" persists even after the collaboration ends, undermining long-term human capability.

Explanations create false confidence that can mislead users: The study (n=160) found that incorrect explanations accompanying correct AI advice compromised human-AI team performance during collaboration, suggesting that people trust explanations even when they shouldn't, without adequate verification mechanisms.

Explainable AI requires quality control, not just transparency: Simply making AI reasoning visible doesn't automatically improve human-AI collaboration. The research demonstrates that transparency without accuracy can actively harm decision-making and knowledge retention, highlighting the need for rigorous validation of explanations before deployment.

Post-collaboration impacts matter for responsible AI design: Beyond immediate task performance, incorrect explanations damage humans' ability to execute tasks independently later. This has serious implications for critical domains like healthcare or finance where users need reliable mental models to make sound decisions.

Current XAI methods need safeguards against misinformation: The findings suggest designers should implement verification systems, confidence indicators, or alternative explanations rather than assuming that any explanation improves human understanding of AI systems.