Explainable AI for Clinical Outcome Prediction: A Survey of Clinician Perceptions and Preferences

Clinicians have diverse preferences for AI explanations: The study surveyed 32 practicing clinicians and found they valued different explanation methods for the same AI predictions, suggesting that a "one-size-fits-all" approach to explainable AI in healthcare is insufficient and organizations should offer multiple explanation options.

Free-text explanations from AI systems require careful validation: While LLM-generated free-text rationales were intuitive for clinicians to understand, the study highlights potential limitations with this approach, implying that AI-generated explanations need human oversight to ensure accuracy before clinical use.

Context matters for choosing explanation methods: The research provides specific recommendations for when different techniques (LIME, attention highlights, exemplar retrieval, and LLM rationales) are more appropriate, indicating that healthcare organizations should match explanation methods to specific clinical contexts rather than applying them universally.

Explainability alone doesn't guarantee clinical integration: The study examined not just whether clinicians understood AI explanations, but whether they could actually integrate them into their decision-making workflow, revealing that technical explainability is only one component of responsible AI deployment in clinical settings.

Clinician input is essential for AI system design: By directly surveying practicing clinicians about their preferences and needs, this research demonstrates that responsible AI development in healthcare requires ongoing engagement with end-users rather than relying solely on technical metrics.