Unjustified Sample Sizes and Generalizations in Explainable AI Research: Principles for More Inclusive User Studies

Small, unjustified samples undermine ethical AI claims: The analysis of 220 XAI studies found that most researchers did not provide rationales for their sample sizes, yet still drew broad conclusions about whether AI systems meet ethical requirements for explainability. This means claims about "responsible AI" may be based on inadequate evidence.

Findings don't generalize beyond who was tested: Most papers generalized their conclusions to broader populations than their actual participants, and quantitative studies with larger samples were no more likely to make broader claims than smaller studies. This suggests researchers may be overstating how well their findings apply to diverse users with different needs.

Explainability needs vary by person: The paper emphasizes that "different people may have different explanatory needs," yet most XAI research doesn't adequately represent diverse populations. This means current explainability solutions may work well for some users but fail others—undermining the goal of inclusive, ethical AI.

Methodological gaps weaken accountability: These research practices "can impede evaluations of whether XAI systems implement the explainability called for in ethical frameworks," meaning companies and regulators cannot reliably verify that AI systems actually meet their stated ethical commitments.

Better standards are needed: The authors outline principles for more rigorous XAI user studies, suggesting the field needs clearer methodological guidelines to ensure explainability research produces trustworthy evidence about whether AI systems truly serve diverse users fairly.