Mitigating belief projection in explainable artificial intelligence via Bayesian Teaching

People assume AI thinks like them: Without explanation, participants predicted the AI would make the same classification decisions as they would. This "belief projection" is a natural but often incorrect assumption that can lead to misplaced trust in AI systems.

Tailored explanations can correct misconceptions: Explanations generated using Bayesian Teaching—which are specifically designed to shift people's understanding toward accurate predictions of the AI's actual behavior—successfully moved participants away from their incorrect assumptions and improved their ability to predict what the AI would actually do.

Different explanation types serve different purposes: The research found that breaking down explanations into sub-examples (saliency maps) helped people better detect errors in familiar categories, while showing whole examples was more effective for helping people understand how the AI would handle unfamiliar cases they hadn't seen before.

Explanation design matters for responsible AI deployment: Rather than using generic explanations, this work suggests that effective XAI should explicitly account for how humans naturally reason about unfamiliar systems and be tailored to correct predictable misunderstandings, which is crucial for building appropriate user trust and enabling better human oversight of AI systems.