Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven decision support

Current explainable AI approaches may not improve decision-making as intended. The paper argues that despite efforts to make AI systems more transparent, people often don't engage sufficiently with explanations to actually benefit from them, suggesting the fundamental approach needs reconsideration.

People struggle with AI recommendations for two key reasons: loss of control and misalignment with how humans naturally think. The paper identifies that traditional recommendation systems undermine human agency and don't match the cognitive processes people actually use when making decisions, which contributes to either over-reliance or dismissal of AI suggestions.

A new "Evaluative AI" framework could better support human decision-making by providing evidence rather than recommendations. Instead of telling people what to do, this machine-in-the-loop approach would present evidence for and against different options, allowing humans to maintain decision-making authority while still benefiting from AI insights.

The history of decision support systems shows that simply providing information doesn't guarantee better outcomes. Early research revealed people either ignored recommendations due to distrust or followed them blindly without critical evaluation—problems that adding explanations alone hasn't fully solved.