Towards Human-centered Design of Explainable Artificial Intelligence (XAI): A Survey of Empirical Studies

The "XAI Gap" is Real: Despite significant advances in explainable AI algorithms, successful real-world implementations remain rare because there is often a mismatch between what XAI systems are technically designed to do and how end-users actually perceive and use these explanations.

One Size Does Not Fit All: The survey identifies diverse stakeholders with different needs—such as doctors, judges, patients, and defendants—requiring tailored explanation approaches rather than universal XAI solutions. Different users need different types of explanations depending on their role and context.

Human-Centered Testing is Essential: The paper emphasizes that explainability is fundamentally a human property, not just a technical one. Empirical studies involving actual users are critical to understanding whether explanations truly help people understand, trust, and adopt AI systems—moving beyond theoretical assumptions.

Evaluation Metrics Must Match Goals: The survey highlights that how we measure XAI success depends on what we're trying to achieve. Different evaluation metrics are needed depending on whether the goal is to build trust, improve understanding, enable better decision-making, or ensure fairness.

Common Pitfalls Exist in Current Research: The paper identifies recurring mistakes and limitations in existing human-centered XAI studies, suggesting the field needs to learn from these patterns to design more effective explanations for high-stakes applications like medicine and criminal justice.