ExplainabilityTransparencyTrustworthinesscs.AI
Visualizations for an Explainable Planning Agent
Tathagata Chakraborti, Kshitij P. Fadnis, Kartik Talamadupula, Mishal Dholakia, Biplav Srivastava, Jeffrey O. Kephart, Rachel K. E. Bellamy
September 13, 2017
AAAI Fall Symposia
9 citations
Key Takeaways
-
Visualization of internal decision-making is essential for trust: The paper emphasizes that making an AI planning agent's reasoning process visible—from sensory inputs through to final decisions—is "a crucial step towards achieving" trust and establishing "common ground with the end-to-end automated planning system."
-
Explainable AI Planning can reframe visualization as an explanation problem: The research demonstrates that planners can leverage explainable planning techniques to present plans as concise, model-based visualizations, making the agent's reasoning more interpretable to human decision-makers.
-
Human-in-the-loop systems require transparency: The paper stresses that "imposing transparency and explainability requirements" on automated planning agents is "especially important" when humans need to make decisions alongside or based on the agent's recommendations, supporting collaborative rather than fully autonomous decision-making.
-
Real-world application in smart assistants: The authors demonstrate these visualization capabilities in a practical context—an instrumented meeting space with a smart assistant—suggesting that explainable planning visualization has concrete applications beyond theoretical research.
-
Externalizing agent reasoning supports accountability: By making visible the agent's "brain" and decision progression, these visualizations enable humans to understand and potentially challenge the agent's choices, supporting responsible deployment of automated planning systems.