Can counterfactual explanations of AI systems' predictions skew lay users' causal intuitions about the world? If so, can we correct for that?

Counterfactual explanations can mislead users about how the world works: When AI systems use counterfactual explanations (e.g., "if your income were $5,000 higher, you'd be approved for a loan"), people incorrectly begin to believe these factors are actually causing real-world outcomes, even though AI systems only detect correlations in data, not true causal relationships.

The problem is significant and measurable: Across 364 study participants, researchers found that exposure to counterfactual AI explanations unjustifiably shifted people's causal beliefs about the factors the AI considered—a concerning finding for any organization deploying explainable AI systems to the public.

A simple warning can help: The good news is that the effect can be partially corrected. When users were explicitly told that "AI systems capture correlations and not necessarily causal relationships," their tendency to misinterpret AI explanations as causal claims was reduced, suggesting that clear disclaimers matter.

Explainability tools need responsible design: This research highlights that making AI "more transparent" doesn't automatically make it more trustworthy or helpful—poorly designed explanations can actually distort people's understanding of real-world cause-and-effect, potentially leading to flawed decision-making.

Responsible AI requires cognitive science input: The findings underscore why AI developers should consult psychology and cognitive science research when designing explanations, not just focus on technical accuracy.