Scientific Explanations in Health Sciences: Causality, Trust, and Epistemic Adequacy

Explainability in medical AI requires more than technical transparency: The paper argues that current approaches to making AI decisions understandable in healthcare are missing crucial insights from philosophy of science and medicine about what actually constitutes a good explanation, suggesting that simply showing how an AI model works is insufficient for clinical contexts.

Causality matters more than correlation for medical trust: Medical professionals and patients need to understand why an AI makes a recommendation through causal reasoning (what actually causes a disease), not just statistical patterns the model detected, which is essential for building appropriate trust in high-stakes healthcare decisions.

Different stakeholders need different explanations: The paper identifies that doctors, patients, hospital administrators, and regulators have different pragmatic needs and levels of expertise, meaning a one-size-fits-all explanation approach to medical AI will fail to adequately serve diverse groups who need to understand and trust these systems.

Philosophy of science has been overlooked in AI explainability research: Decades of philosophical work on what makes explanations adequate in medicine and science have been largely ignored by AI researchers developing explainability methods, leaving the foundational assumptions of XAI insufficiently examined and potentially flawed.

Trust in medical AI requires both epistemic and relational dimensions: Building appropriate confidence in AI systems depends not only on understanding the evidence and reasoning (epistemic trust) but also on the relationships and accountability structures between patients, clinicians, and AI developers.