ExplainabilityTransparencyTrustworthinesscs.AI
Contrastive Explanations of Plans Through Model Restrictions
Benjamin Krarup, Senka Krivic, Daniele Magazzeni, Derek Long, Michael Cashmore, David E. Smith
March 29, 2021
Journal of Artificial Intelligence Research
53 citations
3 influential
Key Takeaways
-
Users naturally ask "why this plan, not that one?" questions: The paper's user study found that when people ask about automated plans, they typically frame questions contrastively (e.g., "why did the system choose action A instead of action B?"), rather than asking for general explanations. This insight suggests that explanation systems should be designed to compare alternatives rather than simply justify single decisions.
-
Plan negotiation through iterative questioning improves understanding: A second user study demonstrated that an iterative process—where the system generates hypothetical alternative plans based on user constraints and questions—helps users arrive at satisfactory plans they understand better. This suggests that effective AI explanations may require back-and-forth dialogue rather than one-time explanations.
-
A taxonomy of user questions can guide explanation design: The researchers developed a formal taxonomy of the types of questions users actually ask during plan negotiation. This structured categorization of real user questions provides a practical foundation for building explanation systems that address what people actually want to know, rather than what designers assume they need.
-
Computational complexity is manageable for practical implementation: The paper formally defined and empirically evaluated different constraint compilations used to generate contrastive explanations. The findings suggest these methods are computationally feasible, indicating that contrastive explanation approaches can be practically implemented in real planning systems without prohibitive computational costs.