Explainable AI in Healthcare: to Explain, to Predict, or to Describe?

Explainable AI methods describe model behavior rather than reveal true causes: The paper distinguishes between description (how a model works) and explanation (why it works based on underlying biological mechanisms). Current explainable AI tools excel at the former but fall short on the latter, meaning they show what factors a model considers without proving those factors are causally responsible for outcomes.

Different stakeholders need different types of information: Patients, doctors, AI developers, and medical guideline creators all require different kinds of explanations. The paper emphasizes that a one-size-fits-all approach to explainability in healthcare is insufficient, as what counts as "relevant information" varies significantly by audience and use case.

Explainable AI has limited ability to provide actionable medical advice: Because these methods cannot reliably establish cause-and-effect relationships in biological systems, they should not be trusted as the primary basis for giving patients or doctors specific treatment recommendations. This is a critical limitation for clinical applications where actionable guidance is essential.

Face validity concerns remain unresolved: The paper highlights that explainable AI methods cannot adequately validate whether an AI model's logic aligns with established medical knowledge and biological reality—a key requirement for clinician trust and adoption in healthcare settings.