ExplainabilityTrustworthinessAccountabilitycs.HC
Can AI Explanations Make You Change Your Mind?
Laura Spillner, Rachel Ringe, Robert Porzel, Rainer Malaka
August 11, 2025
arXiv.org
4 citations
Key Takeaways
-
Users often ignore AI explanations: The study found that participants frequently spent little time reading explanations from AI systems, suggesting that the theoretical benefit of explanations for catching AI errors may not materialize in practice if people don't actually engage with them carefully.
-
Careful reading of explanations matters for changing minds: The research demonstrates a direct link between how thoroughly people consider AI explanations and their willingness to question the AI's suggestions—those who read explanations in detail were more likely to be open to reconsidering their initial judgments.
-
Human oversight requires genuine engagement: The paper highlights a critical gap in responsible AI design: explanations alone cannot prevent biased or erroneous AI decisions if users don't examine them in sufficient detail, undermining a key assumption in human-in-the-loop AI systems.
-
Factors influencing explanation engagement remain important to identify: The exploratory analysis investigates what causes some people to read explanations more carefully than others, suggesting that understanding these factors could help designers create systems where users are more likely to engage meaningfully with AI reasoning.
Abstract
In the context of AI-based decision support systems, explanations can help users to judge when to trust the AI's suggestion, and when to question it. In this way, human oversight can prevent AI errors and biased decision-making. However, this rests on the assumption that users will consider explanations in enough detail to be able to catch such errors. We conducted an online study on trust in explainable DSS, and were surprised to find that in many cases, participants spent little time on the explanation and did not always consider it in detail. We present an exploratory analysis of this data, investigating what factors impact how carefully study participants consider AI explanations, and how this in turn impacts whether they are open to changing their mind based on what the AI suggests.