AI2030 Responsible AI Tools Marketplace

Key Takeaways

The system achieved high diagnostic accuracy (90% Top-1, 100% Top-3) compared to traditional machine learning models, suggesting that conversational AI powered by large language models may offer improved performance for early health assessment when evaluated against established baseline approaches.
Explainability and transparency were built into the system's design through Chain-of-Thought prompting, allowing the chatbot to show its reasoning process to users—addressing a key concern that most existing AI health systems lack transparency about how they reach conclusions.
Expert physician evaluation (17 doctors across 14 conditions) found the system generally favorable on safety-related criteria, indicating that medical professionals assessed the conversational quality and diagnostic plausibility as acceptable, though the paper does not specify which safety concerns were prioritized or any limitations identified.
The system uses Retrieval-Augmented Generation to ground responses in medical knowledge, rather than relying solely on the language model's training data, which helps reduce the risk of hallucinated or inaccurate medical information—a critical concern for responsible deployment in healthcare.
The conversational approach addresses a real healthcare gap by making early diagnosis more interactive and accessible, potentially helping users in areas with limited specialist access, though the paper does not discuss how the system would be validated in real-world clinical settings or what safeguards would prevent misuse as a replacement for professional medical care.

Abstract

Healthcare systems around the world are grappling with issues such as inefficient diagnostics, rising costs, and limited access to specialists. These challenges often contribute to delays in treatment and poorer health outcomes. Most existing AI and deep learning based health assessment systems offer limited interactivity and transparency, reducing their usefulness for user-centered health support. This research introduces a conversational chatbot powered by a Large Language Model (LLM), using GPT-4o, Retrieval-Augmented Generation, and explainable AI techniques. The chatbot engages users in a dynamic conversation to extract and normalize symptoms while identifying and ranking potential health conditions through similarity matching and adaptive questioning. Using Chain-of-Thought prompting, the system also provides more transparent explanations of its reasoning process. When evaluated against traditional machine learning models, including Naive Bayes, Logistic Regression, SVM, Random Forest, and KNN using both TF-IDF and CountVectorizer feature extraction, the proposed LLM-based system achieved a Top-1 accuracy of 90% and a Top-3 accuracy of 100%. The system was additionally evaluated through a cross-sectional expert evaluation involving 17 physicians across all 14 conditions, with the results indicating generally favorable assessments of conversational quality, early diagnostic plausibility, and safety-related criteria. These findings demonstrate the potential of explainable conversational AI as a health and well-being support tool for early symptom assessment. However, the proposed system is not intended for clinical diagnosis or clinical decision-making, and further validation would be required before any use in healthcare practice.

Key Takeaways

The system achieved high diagnostic accuracy (90% Top-1, 100% Top-3) compared to traditional machine learning models, suggesting that conversational AI powered by large language models may offer improved performance for early health assessment when evaluated against established baseline approaches.
Explainability and transparency were built into the system's design through Chain-of-Thought prompting, allowing the chatbot to show its reasoning process to users—addressing a key concern that most existing AI health systems lack transparency about how they reach conclusions.
Expert physician evaluation (17 doctors across 14 conditions) found the system generally favorable on safety-related criteria, indicating that medical professionals assessed the conversational quality and diagnostic plausibility as acceptable, though the paper does not specify which safety concerns were prioritized or any limitations identified.
The system uses Retrieval-Augmented Generation to ground responses in medical knowledge, rather than relying solely on the language model's training data, which helps reduce the risk of hallucinated or inaccurate medical information—a critical concern for responsible deployment in healthcare.
The conversational approach addresses a real healthcare gap by making early diagnosis more interactive and accessible, potentially helping users in areas with limited specialist access, though the paper does not discuss how the system would be validated in real-world clinical settings or what safeguards would prevent misuse as a replacement for professional medical care.

Abstract

Towards Explainable Conversational AI for Early Diagnosis with Large Language Models

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links

Towards Explainable Conversational AI for Early Diagnosis with Large Language Models

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links