A Diachronic Perspective on User Trust in AI under Uncertainty

Overconfident AI systems cause lasting damage to user trust: The research found that even a small number of incorrect predictions made with high confidence significantly undermines user trust, and this trust recovers very slowly afterward—suggesting that first impressions and early errors have outsized importance in how people evaluate AI systems.

Miscalibration has different impacts depending on the type of error: The study distinguished between two types of miscalibration—when AI is confidently wrong versus unconfidently correct—and found they harm user trust in different ways, indicating that how an AI expresses uncertainty matters as much as whether it's accurate.

Poor AI calibration directly reduces collaboration effectiveness: Beyond just affecting user attitudes, the research demonstrated that degraded trust in an AI system actually reduces the success of human-AI teamwork, meaning that calibration problems have real consequences for task performance, not just user satisfaction.

Users build mental models based on confidence signals and explanations: The paper emphasizes that people don't just evaluate AI on accuracy alone—they form judgments based on how the system presents its confidence level and explains its decisions, making transparency and honest uncertainty communication critical design features for trustworthy AI.