Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant

Conversational interfaces improve understanding but don't solve overreliance: While conversational XAI interfaces led to better user understanding and higher trust compared to traditional XAI dashboards, users of both interface types showed "clear overreliance on the AI system"—suggesting that making explanations more engaging doesn't automatically lead to appropriate skepticism.

The "illusion of explanatory depth" is a key problem: The researchers identified that both conversational and dashboard-based XAI interfaces create a false sense of understanding, where users feel they comprehend the AI system better than they actually do, leading them to trust it too much.

Advanced AI explanations can make overreliance worse: When conversational interfaces were enhanced with large language model (LLM) agents, the overreliance problem actually "amplified," indicating that more sophisticated or natural-sounding explanations may paradoxically increase inappropriate trust in AI systems.

Current XAI design needs to prioritize appropriate reliance: The findings suggest that responsible AI design should focus not just on helping users understand AI systems, but specifically on calibrating their reliance—ensuring users trust AI appropriately rather than blindly, which requires rethinking how explanations are presented.