What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks

XAI evaluation requires multiple validation approaches, not just user studies: While user testing is important for understanding how people respond to explanations, the paper argues that the technical mechanisms generating those explanations need separate, algorithm-focused validation. This means responsible AI requires both technical auditing and real-world user feedback.

XAI systems have modular components that need individual assessment: The authors identify three distinct layers—technical building blocks, user-facing explanations, and communication protocols—that should each be evaluated separately. This modular approach helps identify which specific part of an XAI system is working well or poorly, rather than treating it as a black box.

Context and use case matter critically for XAI validation: The framework emphasizes that evaluation must account for "the anticipated use case," meaning an explanation system that works well in one setting (e.g., healthcare) may not work in another (e.g., criminal justice). There is no one-size-fits-all standard for good explanations.

Current XAI evaluation practices are incomplete: The paper identifies that existing evaluation methods are "elusive and challenging" and often over-rely on single evaluation facets, suggesting that many deployed XAI systems today may not have been thoroughly validated across all necessary dimensions before real-world use.

Responsible AI requires "sociotechnical" thinking: The framework recognizes that XAI isn't purely a technical problem—it involves social communication, human understanding, and organizational context. Evaluating responsible AI means considering engineering, social, and human factors together.