ExplainabilityTransparencyTrustworthinesscs.AI
Explainers' Mental Representations of Explainees' Needs in Everyday Explanations
Michael Erol Schaffer, Lutz Terfloth, Carsten Schulte, Heike M. Buhl
November 13, 2024
arXiv.org
Key Takeaways
-
Explainers naturally adapt explanations based on evolving understanding of what the listener knows: The study found that when people explain technology, they start with vague assumptions about what the listener understands, but these assumptions solidify into "strong beliefs" as the explanation progresses. This suggests AI systems should similarly update their understanding of users' knowledge levels rather than providing static explanations.
-
Effective explanations require two complementary perspectives: According to the research's "dual nature theory," good explanations of technological artifacts need to cover both "Architecture" (observable, measurable features) and "Relevance" (interpretable aspects that matter to the user). Current XAI systems may be missing this balance.
-
Users' knowledge needs shift during explanations: The study observed that explainers initially assume users need information about Architecture (how something works mechanistically), but recognize over time that users also need to understand Relevance (why it matters). AI explanation systems should similarly recognize and respond to these shifting informational needs rather than treating explanation as a one-time event.
-
Responsible AI requires modeling user needs dynamically: The research emphasizes that explainers maintain "mental representations" of what users need to know, which update continuously. For AI systems to be truly responsible and user-centered, they need built-in components that track and adapt to individual users' developing understanding and interests.