ExplainabilityTransparencyTrustworthinesscs.CL
A Measure of Explanatory Effectiveness
Dylan Cope, Peter McBurney
May 20, 2023
arXiv.org
Key Takeaways
-
The explainee matters more than we think: The paper highlights that most AI explanation research ignores the person receiving the explanation, despite explanation being fundamentally about communication between two parties. This suggests current explanation methods may not actually be effective at helping people understand AI systems.
-
Explanations should be measured by their impact on understanding: Rather than assessing explanations based on technical metrics alone, the paper proposes measuring explanatory effectiveness by tracking changes in the explainee's internal understanding. This means a good explanation is one that actually changes how someone comprehends an AI system.
-
Explanation is a cooperative process: By framing explanation as a two-player game where both the AI system and the person asking questions work together, the paper suggests that effective explanations require active participation from both sides, not just one-way information delivery from the AI.
-
We need automated ways to assess explanation quality: The paper proposes developing automated assessment methods for explanations, which could help ensure that AI systems generate genuinely helpful explanations rather than technically correct but confusing ones—an important step toward more trustworthy AI systems.
Abstract
In most conversations about explanation and AI, the recipient of the explanation (the explainee) is suspiciously absent, despite the problem being ultimately communicative in nature. We pose the problem `explaining AI systems' in terms of a two-player cooperative game in which each agent seeks to maximise our proposed measure of explanatory effectiveness. This measure serves as a foundation for the automated assessment of explanations, in terms of the effects that any given action in the game has on the internal state of the explainee.