XEQ Scale for Evaluating XAI Experience Quality

XAI evaluation needs to go beyond single explanations: The paper argues that current methods for assessing AI explanations are too narrow, focusing on one-time interactions. Instead, users benefit from multiple explanations over time that they can engage with repeatedly, requiring new evaluation approaches.

Four key dimensions define a good XAI experience: The XEQ Scale measures quality across learning (did users understand the AI's decision?), utility (was the explanation useful?), fulfilment (did it meet user needs?), and engagement (was the interaction compelling?). This multi-dimensional approach better captures what makes explanations valuable to real people.

User-centered evaluation is now measurable: The paper provides a validated tool that organizations can use to assess whether their AI explanation systems actually work for the people using them, moving beyond technical metrics to focus on human experience and satisfaction.

Expert validation supports the framework's credibility: The XEQ Scale was tested with XAI experts for content validity and validated through a large-scale pilot study, providing evidence that it reliably measures what it claims to measure across different user groups and contexts.

This addresses a gap in responsible AI practice: While many organizations are adding explanations to their AI systems, they often lack systematic ways to evaluate whether these explanations actually help users make better decisions or understand AI limitations—the XEQ Scale provides that missing measurement capability.