Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making

Humans often passively accept AI recommendations without critical thinking. The paper identifies that in typical AI-assisted decision-making, humans rarely engage in analytical thinking when reviewing AI suggestions, simply accepting or rejecting them wholesale rather than engaging deeply with the reasoning.

A new "deliberation" approach helps humans and AI discuss disagreements constructively. The proposed Human-AI Deliberation framework breaks down decisions into specific dimensions, allowing humans and AI to discuss conflicting opinions point-by-point rather than treating the AI's recommendation as a take-it-or-leave-it proposition.

Large language models can facilitate more nuanced human-AI communication. By using LLMs as an intermediary between humans and specialized AI models, the system enables more flexible, conversational interactions that help humans articulate the "nuances of conflicting opinions" they struggle to communicate in traditional AI interfaces.

Deliberative AI improved human decision-making better than standard explainable AI. In evaluation on graduate admissions decisions, the deliberative approach outperformed conventional explainable AI (XAI) tools in helping humans develop "appropriate reliance" on AI—meaning they trusted it when they should and questioned it when warranted—and improved overall task performance.

This design promotes human agency and reflection in AI-assisted decisions. Rather than positioning humans as passive reviewers, the deliberation framework treats them as active participants in a discussion, encouraging the analytical thinking and informed judgment necessary for responsible AI-assisted decision-making.