How model accuracy and explanation fidelity influence user trust

Accuracy trumps explainability: The study found that model accuracy is more important for building user trust than providing explanations, suggesting that getting the right answer matters more to users than understanding how the system arrived at that answer.

Poor explanations can backfire: Adding explanations to classification results can actually harm user trust when those explanations are nonsensical or low-quality, indicating that a bad explanation is worse than no explanation at all.

Users are resistant to deception: High-fidelity explanations (those that accurately represent how the model works) cannot trick users into trusting an inaccurate classifier, demonstrating that users maintain some skepticism even when presented with convincing explanations.

Trust is more complex than users realize: The research revealed a mismatch between users' implicit (observed) trust and their explicit (self-reported) trust, suggesting that what users say about their trust in AI systems may not align with how they actually behave, which has important implications for assessing real-world AI adoption.

Explainability alone is insufficient for responsible AI: The findings challenge the assumption that making AI systems more explainable automatically improves user trust and acceptance—responsible AI deployment requires prioritizing accuracy and ensuring explanation quality, not just adding explanations.