ExplainabilityTransparencyTrustworthinesscs.HC
How Can Explainable Artificial Intelligence Improve Trust and Transparency in Medical Diagnosis Systems?
Altynbek Seitenov, Ainur Nurzhanova, Azhar Bekbussinova, Yerassyl Bolatkan
March 14, 2026
arXiv.org
Key Takeaways
-
Explanations make AI more trustworthy in medical settings: The study found a meaningful positive correlation (r = 0.48, p = 0.01) between understanding how AI works and trusting its diagnostic recommendations, suggesting that transparency directly addresses clinician concerns about black-box medical AI systems.
-
Doctors want AI as a partner, not a replacement: Despite increased trust from explanations, participants still preferred AI to function as a support tool rather than replace human clinical judgment, indicating that responsible AI deployment in healthcare should enhance rather than substitute physician decision-making.
-
Understanding XAI improves perceived usefulness: Knowledge of explainable AI techniques showed a strong positive correlation with perceived usefulness (r = 0.60, p = 0.001), suggesting that educating healthcare professionals about how AI explanations work can increase their confidence in adopting these tools.
-
Transparency addresses a critical adoption barrier: The research indicates that many existing AI diagnostic systems operate as "black boxes," limiting clinicians' ability to understand decisions—a fundamental barrier to adoption that explainability directly addresses by improving clarity and perceived safety of AI recommendations.
Abstract
The growing adoption of artificial intelligence in healthcare has raised concerns about the transparency and trustworthiness of AI-driven medical diagnosis systems. Many existing models operate as black boxes, limiting clinicians' ability to understand how decisions are made. Explainable Artificial Intelligence (XAI) has been proposed as a solution to improve transparency, interpretability, and trust in AI-assisted medical tools. This study investigates the relationship between explainability and trust in AI-based diagnostic systems. A structured survey of 30 medical students was conducted to examine the influence of XAI understanding, confidence in AI decisions, perceived usefulness, and adoption intentions. The results indicate that explanations significantly increase trust, clarity, and perceived safety of AI recommendations. Knowledge of XAI showed a positive correlation with trust (r = 0.48, p = 0.01) and perceived usefulness (r = 0.60, p = 0.001). The findings suggest that explainability is a key factor for successful integration of AI in healthcare decision support systems. While AI explanations improve transparency and trust, participants still prefer AI to function as a support tool rather than replacing human clinical judgment.
15 pages, 22 figures, survey study on explainable AI in healthcare decision support systems
Paper Info
arXiv ID2604.16340
PublishedMarch 14, 2026
Citations
0
Relevance88%
VenuearXiv.org
TypeJournalArticle, Review
Categorycs.HC
Responsible AI Topics
Explainability