Explanation as Question Answering based on a Task Model of the Agent's Design

Explanations should be grounded in how AI systems are actually designed: The paper demonstrates that effective explanations come from explicitly documenting an AI agent's tasks, goals, and methods through a "Task-Method-Knowledge model"—essentially creating a formal blueprint of the system's design that can be used to answer user questions.

Ask users what they actually want to know: Rather than deciding what explanations to provide, the researchers used participatory design with focus groups to collect real questions people had about an AI agent's operations. This human-centered approach ensures explanations address genuine user concerns rather than what designers assume people need.

Explanations work best as question-answering systems: The paper shows that embedding a question-answering agent (AskJill) within an AI system allows users to get targeted explanations about specific aspects—like how recommendations are produced or what vocabulary the system uses—rather than receiving generic explanations that may not match individual needs.

Transparency requires documenting system vocabulary and mechanisms: The approach emphasizes that users need clear explanations not just of what an AI system does, but of the specific terms, knowledge, and mechanisms it uses. In the Skillsync example, this means explaining both the system's tasks and the particular vocabulary it employs for matching workers to upskilling opportunities.

Design documentation can serve dual purposes: By formalizing how an AI agent is designed, organizations create both a technical specification and a foundation for generating user-facing explanations—making transparency an integrated part of system design rather than an afterthought.