ExplainabilityTransparencyTrustworthinesscs.AI
Provenance-Based Assessment of Plans in Context
Scott E. Friedman, Robert P. Goldman, Richard G. Freedman, Ugur Kuter, Christopher Geib, Jeffrey Rye
November 3, 2020
arXiv.org
2 citations
Key Takeaways
-
Plans depend on multiple information sources with varying reliability: The paper demonstrates that automated plans often rely on diverse external data sources and agents of different trustworthiness levels. By tracking where plan information comes from (its "provenance"), decision-makers can now understand which parts of a plan are based on highly reliable sources versus less certain ones.
-
Transparency about plan assumptions enables better human oversight: The approach makes explicit the assumptions and dependencies embedded in automated plans, allowing human reviewers to understand what would happen if certain information proved incorrect. This supports what the authors call "counter-factual assessment"—essentially asking "what if this assumption was wrong?"—which is critical for responsible deployment of planning systems.
-
Risk and confidence can be systematically traced through complex plans: Rather than treating a plan as a black box, this method propagates information about reliability and risk through the entire plan structure, enabling assessment of a plan's "sensitivity" (which parts are most vulnerable to information changes) and overall confidence level.
-
Existing technical standards can represent plan provenance: By using the established PROV-O standard for representing information provenance, this approach makes plan explanations more standardized and potentially interoperable across different planning systems, supporting broader adoption of transparent AI planning.