Towards Better User Requirements: How to Involve Human Participants in XAI Research

Current XAI research often fails to properly involve actual users: The paper identifies significant limitations in how existing XAI studies incorporate human participants when designing explainability systems, suggesting that many projects don't adequately understand who will actually use these systems or what they need.

User requirements are frequently assumed rather than discovered: Researchers often identify what users need based on assumptions rather than through direct engagement with real users, which can lead to XAI systems that don't address genuine user concerns or problems.

Human-Centered XAI requires intentional methodological approaches: The paper proposes that deriving better user requirements demands specific, deliberate methods for involving human participants—suggesting this isn't something that happens automatically in typical AI research workflows.

Bridging the gap between technical capability and user needs is critical: By highlighting the gap between how XAI systems are currently built and what users actually require, the paper emphasizes that technical explainability alone isn't sufficient without understanding the human context in which these systems operate.

Responsible AI development depends on participatory design practices: The paper's recommendations suggest that creating trustworthy and useful AI systems requires researchers to actively involve diverse human participants throughout the design process, not just in final testing phases.