Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond

Explainability is a barrier to clinical adoption: The paper identifies "insufficient explainability and transparency in most existing AI systems" as "one of the major reasons that successful implementation and integration of AI tools into routine clinical practice are uncommon," highlighting that performance alone is not enough for healthcare AI to be trusted and used by clinicians.

Deep learning's "black box" problem is especially critical in medicine: While deep neural networks deliver strong performance, they cannot easily explain their decisions—a particular concern in healthcare where clinicians need to understand why an AI system made a recommendation before trusting it with patient care.

Multi-modal and multi-centre data fusion can improve explainability: The authors propose combining multiple types of data (multi-modal) from different healthcare centers (multi-centre) as a solution to make AI decision-making more transparent and interpretable, validated through real clinical scenarios.

Explainable AI (XAI) is becoming essential, not optional: The paper emphasizes that XAI is "more and more crucial for deep learning powered applications, especially for medical and healthcare studies," suggesting that transparency should be a core requirement rather than an afterthought in medical AI development.

Validation through real clinical scenarios matters: The authors demonstrate their XAI solutions through actual clinical use cases with "comprehensive quantitative and qualitative analyses," suggesting that explainability solutions must be tested in realistic healthcare contexts to prove their value.