Evaluating Actionability in Explainable AI

Explanations alone don't guarantee usefulness: The paper challenges a fundamental assumption in XAI that simply providing explanations to users will lead to meaningful action. The researchers found that the connection between what explanations show and what users actually do with them is often unclear and untested by AI creators.

12 types of information can drive 60 different user actions: Through interviews with professionals in education and medicine, the study identified a catalog showing that users rely on 12 distinct categories of information to take approximately 60 different actions. This specificity helps move beyond vague notions of "understanding" to concrete, measurable outcomes.

AI creators should test their assumptions about how explanations will be used: Rather than assuming explanations will be helpful, the paper recommends that AI developers explicitly articulate how they expect users to act on the information provided and then validate these assumptions through testing with actual users.

Current XAI systems may not be designed with real-world actions in mind: The research suggests that many existing explainable AI systems are built without clearly connecting their explanations to the specific decisions or actions users need to take in fields like education and medicine, potentially limiting their practical value.

The catalog expands possibilities for better XAI design: By mapping information types to user actions, the paper provides a framework that can help designers create explanations tailored to support the specific, actionable outcomes that end users actually need in their work.