Keep Your Friends Close and Your Counterfactuals Closer: Improved Learning From Closest Rather Than Plausible Counterfactual Explanations in an Abstract Setting

Simpler explanations may be better than "realistic" ones: The study found that users learned more effectively from counterfactual explanations showing the smallest changes needed to alter an AI decision, rather than explanations designed to be "computationally plausible" (realistic-sounding). This suggests that AI explanations don't need to mimic real-world constraints to be helpful.

There's a gap between what technologists think is plausible and what users actually find clear: The research reveals a disconnect between "computational plausibility" (what AI developers design) and "psychological plausibility" (what users actually perceive as understandable). Users found the simpler explanations more intuitive, even though developers might consider them unrealistic.

User experience didn't differ, but learning outcomes did: While both groups reported similar satisfaction with the explanations, the group receiving simpler counterfactuals actually performed better at the learning task. This suggests that subjective comfort with an explanation doesn't guarantee it's actually helping people understand AI decisions.

Current AI explanation design may prioritize the wrong goals: The findings challenge the assumption that making AI explanations more "realistic" automatically improves their value to users. Designers may need to reconsider whether computational plausibility should be a primary design constraint for explainable AI systems.