The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies

Lack of transparency is a major barrier to AI adoption in healthcare: Clinicians need confidence that AI systems can be trusted before implementing them in clinical practice, and explainability is identified as a key solution to overcome this implementation gap.

The reason for demanding explanations should drive design choices: What gets explained depends on why explainability is needed, which then determines which properties matter most (interpretability versus fidelity) and which explanation methods are most appropriate for a given situation.

Different types of explanations serve different purposes: The paper identifies multiple approaches—explainable modeling versus post-hoc explanations, model-based versus attribution-based versus example-based methods, and global versus local explanations—and practitioners should choose based on their specific needs rather than assuming one approach fits all.

Evaluation of AI explanations lacks standardization: Quantitative metrics for objectively measuring whether explanations are actually clear and understandable are still missing for some explanation types (particularly example-based methods), making it difficult to compare and validate different approaches.

Explainability alone is not sufficient for trustworthy AI: While explainable AI is an important step toward trustworthiness in healthcare, the paper concludes that the real-world benefits of explainability still need to be proven in practice and may need to be combined with other complementary measures.