Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AI

One-size-fits-all XAI doesn't work: The paper demonstrates that explainable AI systems need to be evaluated differently depending on their intended use—whether for debugging AI models or supporting human decision-making—rather than applying universal evaluation standards across all applications.

Current XAI research often misses real-world needs: Algorithmic researchers developing XAI tools frequently don't consider how these tools will actually be used in practice, which can lead to ineffective systems or unintended negative consequences for end users, highlighting a disconnect between academic development and practical deployment.

Different users have different requirements: Survey findings from both XAI experts and general workers reveal that the criteria for what makes an explanation "good" varies significantly depending on the usage context, meaning practitioners need guidance tailored to their specific use case rather than generic recommendations.

Evaluation methods must be context-aware: The paper advocates for developing evaluation approaches that account for user requirements specific to each usage context, rather than relying on standardized metrics that may not capture what matters most for a particular application.

Responsible AI requires usage-informed practices: The research calls for a shift in how XAI is developed and assessed—moving toward evaluation methods that are informed by actual user needs and real-world deployment contexts to ensure AI systems are both technically sound and genuinely useful.