Quantitative and Qualitative Evaluation of Explainable Deep Learning Methods for Ophthalmic Diagnosis

Clinical validation matters: The study evaluated 13 different explainability methods specifically with ophthalmologists rather than relying on standard machine learning benchmarks, finding that Deep Taylor expansion was rated highest (3.85/5) by clinicians for explaining retinal disease diagnoses in OCT images.

Explainability is a barrier to AI adoption in medicine: The researchers identified that "lack of explanations for the decisions made by algorithms...has hampered their acceptance by the clinical community despite highly accurate results," suggesting that accuracy alone is insufficient for clinical deployment.

Not all explanation methods are equally useful for medical imaging: The study found meaningful differences in how clinicians rated various attribution methods, with Deep Taylor, Guided backpropagation, and SHAP performing notably better than others—indicating that explanation quality varies significantly depending on the method chosen.

Clinician feedback is essential for responsible AI development: By incorporating a panel of 14 clinicians to evaluate explanations rather than using automated metrics, the research demonstrates that human expert judgment should guide which AI explanation methods are adopted in clinical settings.

Explainability can increase transparency for diagnosis: The conclusion suggests that when deep learning models can explain their reasoning through appropriate attribution methods, they become more transparent tools that clinicians can better understand and potentially trust for medical decision-making.