Invisible Users: Uncovering End-Users' Requirements for Explainable AI via Explanation Forms and Goals

End-users' needs are being ignored in XAI design: The paper reveals that non-technical end-users—the actual people who rely on AI explanations for important decisions—are not involved in developing explainable AI systems, making these tools potentially ineffective or even harmful in high-stakes fields like healthcare, criminal justice, and finance.

Different explanation types serve different purposes: The study identified that end-users need various explanation approaches (feature-based, example-based, and rule-based explanations), suggesting that a one-size-fits-all explanation method won't work for diverse real-world applications and user needs.

User requirements should directly shape AI development: The EUCA study with 32 layperson participants uncovered specific user requirements and "explanation goals" that researchers can use to directly inspire new XAI algorithms and evaluation metrics, bridging the gap between what technologists build and what users actually need.

Current XAI evaluation may miss what matters most: The paper suggests that how XAI systems are currently tested and evaluated doesn't align with what end-users actually require to make safe, informed decisions, potentially creating a false sense of transparency.

Centering end-users is essential for trustworthy AI: By incorporating non-technical users' requirements into XAI design, the research supports creating AI systems that are more accessible, safer, and more accountable—critical for maintaining public trust in AI-assisted decision-making.