Questioning the AI: Informing Design Practices for Explainable AI User Experiences

The gap between XAI research and real-world design is significant: While algorithmic explainability work has flourished, the paper found that practitioners working on actual AI products face unmet needs, suggesting that much academic XAI research may not address what users actually need to understand.

Users ask diverse, context-specific questions about AI systems: The researchers developed an "XAI question bank" representing prototypical questions users might ask (such as "Why did the AI make this decision?" or "When will this AI fail?"). This demonstrates that explainability needs vary greatly depending on the user's role and context, rather than being one-size-fits-all.

Design practitioners lack adequate tools and frameworks: By interviewing 20 UX and design professionals, the study revealed that current XAI approaches don't provide sufficient guidance for practitioners trying to build explainable AI products in practice, indicating a need for more design-focused resources.

User-centered approaches are essential for XAI: The paper emphasizes that effective explainability requires understanding what specific users need to know in their particular contexts, rather than simply adding technical explanations to AI systems—a shift from algorithm-focused to human-focused design.

The XAI question bank offers a practical design tool: The extended question bank developed in this research can help design teams identify and address user needs for explainability early in the product development process.