Measuring Perceived Trust in XAI-Assisted Decision-Making by Eliciting a Mental Model

Trust in AI explanations is measurable and predictable: The study demonstrates that perceived trust in XAI systems can be quantified through users' mental models, and these quantified values correlate with whether medical experts actually trust or distrust the AI model's recommendations—suggesting trust isn't just subjective feeling but can be systematically assessed.

Explanation quality matters more than accuracy alone: Medical experts' trust was influenced by how well the AI explained its reasoning, not just whether it made correct predictions. The research found that explanation satisfaction attributes directly shaped whether experts relied on the system, highlighting that transparent reasoning is essential for responsible AI deployment in high-stakes domains like healthcare.

Individual differences in how people process AI explanations are important: By using fuzzy linguistic variables to capture each expert's subjective interpretation of explanation strength, the study reveals that people have different mental models of the same AI system. This suggests that one-size-fits-all explanations may not build appropriate trust across diverse users, and personalized explanation approaches may be necessary.

Trust should be validated against actual task performance: The researchers compared their trust measurements against how well medical experts actually performed diagnostic tasks, establishing that measured trust correlates with real-world decision-making behavior—a critical validation that perceived trust reflects meaningful reliance on the system rather than superficial confidence.