Doctor XAvIer: Explainable Diagnosis on Physician-Patient Dialogues and XAI Evaluation

Explainability methods aren't equally effective: The paper's novel evaluation metric (FAD curve) reveals that integrated gradients significantly outperforms Shapley values at explaining diagnostic decisions, suggesting that organizations implementing AI diagnosis tools should carefully evaluate which explanation method they use rather than assuming all approaches are equivalent.

High performance metrics don't guarantee trustworthiness: While Doctor XAvIer achieved strong F1-scores (0.97 for symptom classification, 0.91 for diagnosis), the paper emphasizes the need for separate evaluation of explanation quality, indicating that accurate predictions alone are insufficient for responsible AI deployment in healthcare—the reasoning must also be transparent and validated.

New evaluation standards are needed for AI explanations: The introduction of the Feature Attribution Dropping (FAD) curve and Normalized Area Under the Curve (N-AUC) metric addresses a gap in how we assess whether AI explanations are actually meaningful, suggesting the field needs better standardized tools to evaluate explainability before deploying these systems in clinical settings.

Dialogue-based diagnosis systems require specialized evaluation: By working with transcribed patient-doctor conversations rather than structured medical records, Doctor XAvIer demonstrates that AI systems operating on real-world clinical communication need tailored evaluation approaches that account for the complexity and variability of natural language in medical contexts.