From Dyad to Triad: Eliciting XAI Requirements in Stroke Rehabilitation

Stroke survivors and caregivers have diverse and sometimes conflicting needs for AI transparency: The study found heterogeneous XAI preferences across just three patient-caregiver pairs, suggesting that one-size-fits-all approaches to explainability in medical AI will likely fail to meet actual user needs in rehabilitation contexts.

Standard requirement-gathering methods don't work for patients with communication disorders: The researchers identified that traditional elicitation approaches are "structurally inadequate" for users with acquired communication disorders like aphasia, highlighting a critical gap in how we currently design AI systems for vulnerable populations.

Scaffolding techniques can help bridge the explainability knowledge gap: Four specific strategies—using analogies to familiar systems, creating depersonalized scenarios, offering binary choices, and allowing extra response time—successfully helped participants without AI expertise articulate their transparency preferences.

Facilitators can inadvertently bias patient responses: The study identified three systematic biases (normative bias, hypothesis confirmation bias, and "presence effect") where the scaffolding methods themselves shaped what participants said, underscoring the importance of reflexive awareness when designing patient engagement processes.

Methodological rigor is essential for trustworthy AI in healthcare: By documenting both successful techniques and their inherent risks, this work provides practical guidelines for practitioners to ethically elicit patient preferences while remaining transparent about the limitations of any elicitation method.