ExplainabilityTransparencyTrustworthinesscs.AI
Explainable Planning
Maria Fox, Derek Long, Daniele Magazzeni
September 29, 2017
arXiv.org
307 citations
13 influential
Key Takeaways
-
AI systems need to explain their reasoning to build human trust: The paper emphasizes that as humans delegate more responsibility to AI systems, transparency about how these systems make decisions becomes essential—not just for cooperation, but as a foundational requirement for establishing trust in high-stakes applications.
-
Planning systems have a unique advantage for explainability: Unlike many "black box" AI algorithms, AI planning systems use model-based representations that can be communicated to users in familiar terms, making them better suited than other AI approaches for providing meaningful explanations of system behavior.
-
There's a significant gap between how AI plans and how humans solve problems: The paper acknowledges that even with model-based planning, there remains a fundamental difference between the way planning algorithms operate and human problem-solving approaches, which must be bridged for effective human-AI collaboration.
-
Explainability supports integrated human-AI problem-solving: Beyond transparency alone, the ability to explain AI planning decisions enables true cooperation between humans and intelligent systems, where both parties can understand and build upon each other's contributions to solving complex problems.
Abstract
As AI is increasingly being adopted into application solutions, the challenge of supporting interaction with humans is becoming more apparent. Partly this is to support integrated working styles, in which humans and intelligent systems cooperate in problem-solving, but also it is a necessary step in the process of building trust as humans migrate greater responsibility to such systems. The challenge is to find effective ways to communicate the foundations of AI-driven behaviour, when the algorithms that drive it are far from transparent to humans. In this paper we consider the opportunities that arise in AI planning, exploiting the model-based representations that form a familiar and common basis for communication with users, while acknowledging the gap between planning algorithms and human problem-solving.