Fewer Than 1% of Explainable AI Papers Validate Explainability with Humans

Most XAI research lacks human validation: Only 0.7% of the 18,254 explainable AI papers analyzed included actual human studies to test whether their methods were truly understandable to people, despite widespread claims about improving human understanding.

There's a significant gap between promises and evidence: While XAI researchers frequently claim their techniques make AI more explainable to humans, fewer than 1% provide empirical proof through human evaluation, suggesting many claims may be unsubstantiated.

The field needs stronger standards: The paper identifies a critical methodological problem in XAI research—the absence of rigorous human-centered validation means we cannot confidently know which explanation methods actually work for real people in practice.

Current practices undermine responsible AI goals: If explainability techniques aren't validated with humans, deploying them in high-stakes applications (healthcare, criminal justice, finance) could create a false sense of transparency without delivering genuine understanding or accountability.

Reproducibility and transparency are essential next steps: The authors provide their search methodology and call for increased human studies in future XAI work, emphasizing that responsible AI development requires evidence-based validation rather than theoretical claims alone.