The Who in XAI: How AI Background Shapes Perceptions of AI Explanations

AI background significantly shapes how people interpret explanations: The study found that people with and without AI expertise perceived the same explanations differently across five dimensions, meaning a one-size-fits-all approach to AI explainability won't work equally well for all users.

Both experts and non-experts showed problematic over-reliance on numerical information, but for different reasons: While both groups demonstrated "unwarranted faith in numbers," the underlying causes differed between them, suggesting that simply adding numbers to explanations could inadvertently increase misplaced trust rather than improve understanding.

AI explanations can backfire despite good intentions: The research warns that well-designed explanations meant to increase transparency can actually have negative consequences, including the potential for harmful manipulation of user trust—a critical concern for responsible AI deployment.

Different user groups find value in unexpected places: Each group discovered utility in explanations beyond their original purpose, indicating that designers cannot predict how diverse audiences will actually use AI explanations in practice.

Design interventions are needed to prevent misuse: The authors propose specific design changes to mitigate risks, emphasizing that responsible AI requires not just opening the "black box" but carefully considering who is opening it and how different audiences will interpret the results.