On Model Reconciliation: How to Reconcile When Robot Does not Know Human's Model?

Robots need better ways to explain themselves when they don't know what humans are thinking. The paper addresses a real-world problem: current AI systems assume they understand human expectations, but humans often update their preferences without telling the robot, creating misunderstandings that explanations can't fix.

A back-and-forth conversation can help robots and humans align their understanding. Rather than the robot providing a one-time explanation, the proposed dialog-based approach lets robots make proposals about why they acted differently than expected, and humans provide feedback to refine the explanation—similar to how humans naturally clarify misunderstandings through discussion.

Shared language is essential for meaningful AI explanations. The approach assumes humans and robots agree on basic concepts (predicates and action descriptions) and communicate truthfully. This highlights that responsible AI systems need clear, common ground about what terms mean before they can effectively explain their decisions.

This work makes AI explanations more realistic and practical. By removing the unrealistic assumption that robots know exactly what humans believe, this research addresses a genuine gap between how current explainable AI systems work in theory and how they would need to work in real-world deployments where human preferences change over time.