ExplainabilityTransparencyTrustworthinesscs.AI
JEDAI: A System for Skill-Aligned Explainable Robot Planning
Naman Shah, Pulkit Verma, Trevor Angle, Siddharth Srivastava
October 31, 2021
Adaptive Agents and Multi-Agent Systems
11 citations
Key Takeaways
-
AI systems can be designed to explain their limitations: JEDAI demonstrates that robots can provide customized explanations to users about errors and constraints, helping people understand not just what a robot can do, but importantly, what it cannot do—a key aspect of responsible AI transparency.
-
User education improves human-AI collaboration: By helping non-experts understand AI planning concepts and robot capabilities through interactive planning, JEDAI shows that responsible AI systems should be designed as educational tools that build user understanding rather than simply accepting commands.
-
Explainability requires bridging technical and intuitive thinking: The system synthesizes task and motion planning with explainable AI to let users create "high-level, intuitive plans" while ensuring feasibility, suggesting that responsible AI should translate between human reasoning and technical constraints rather than forcing users to think like engineers.
-
Preventive design reduces harmful outcomes: By ensuring plans are executable before deployment, JEDAI prevents errors at the source rather than just explaining failures after they occur, demonstrating a proactive approach to responsible AI that prioritizes safety and reliability.
-
Outreach and education are core to responsible AI: The paper's emphasis on designing JEDAI specifically for "non-AI experts" through "outreach and educational efforts" suggests that responsible AI development should include making systems understandable and trustworthy to the broader public, not just technical specialists.