Clinicians' Voice: Fundamental Considerations for XAI in Healthcare

Clinicians want AI tools designed around their actual workflows, not just technical capabilities. The study found that end users are primarily concerned about how AI systems will integrate into existing clinical practice and patient care routines, suggesting that explainability alone is insufficient without practical implementation considerations.

AI education for healthcare professionals is essential for successful adoption. The research identifies clinician training as a "crucial factor for the success of AI in healthcare," indicating that responsible AI deployment requires investment in helping doctors and medical staff understand and appropriately use these tools.

Patient-clinician relationships must remain central to AI design. Clinicians expressed concerns about how AI tools might impact their relationships with patients, highlighting that responsible AI in healthcare cannot focus solely on technical performance but must preserve the human elements of medical care.

Most XAI research lacks real-world input from actual users. The authors note that "most of the current research lacks input from end users, and therefore their practical value is limited," suggesting a significant gap between what AI researchers develop and what healthcare practitioners actually need.

General requirements should be established before testing specific AI tools. Rather than evaluating individual AI systems in isolation, the study advocates for a "holistic and exploratory perspective" to first identify what clinicians fundamentally need from AI products, ensuring tools are built on solid foundational principles.