Forms of Understanding for XAI-Explanations

Understanding in AI explanations has two distinct forms: The paper identifies "enabledness" (knowing how to use or decide something) and "comprehension" (knowing that something is true), both of which can range from shallow to deep. This distinction matters because an AI explanation might help you use a system without actually helping you understand why it works.

Current XAI approaches often lack clear definitions of their goal: The paper emphasizes that explainability research rarely defines what "understanding" actually means, creating a fundamental problem—we may be building explanation systems without agreement on what successful understanding looks like or how to measure it.

Understanding develops gradually and requires both types working together: Explanations typically begin with shallow understanding in a specific domain and can progress toward deeper comprehension and enabledness. Importantly, these two forms are "highly interdependent," meaning effective AI explanations need to build both types of understanding simultaneously, not just one or the other.

Deep understanding is necessary for human agency: The paper argues that achieving deeper levels of both comprehension and enabledness is "a prerequisite for human users to gain agency"—suggesting that surface-level explanations that only enable narrow task completion may leave users unable to make independent judgments or adapt to new situations.

XAI faces unique challenges compared to everyday explanations: Drawing on insights from computer science, linguistics, sociology, philosophy, and psychology, the paper identifies that AI explanations present special difficulties beyond those in normal human communication, though the paper itself notes these challenges require further investigation.