Explainable AI Enhances Glaucoma Referrals, Yet the Human-AI Team Still Falls Short of the AI Alone

Explainable AI improved primary care providers' referral accuracy to 59.9% compared to 50.8% without AI support, demonstrating that AI explanations can meaningfully enhance clinical decision-making for identifying patients who need urgent glaucoma specialist care.

Human-AI teams underperformed compared to AI making decisions alone, suggesting that involving humans in the decision loop can paradoxically reduce overall accuracy—a critical finding for designing AI systems in healthcare where fully autonomous AI might outperform collaborative approaches.

Providers perceived intrinsic explanations (built into the AI model) as more useful and trustworthy than post-hoc explanations, and were more likely to follow AI recommendations when using the intrinsic model, indicating that how explanations are presented significantly influences whether clinicians actually use AI guidance.

AI support reduced clinical challenges without increasing workload or requiring additional time, suggesting that well-designed explainable AI can be practically integrated into existing workflows without burdening already-stretched primary care providers.

The performance gap between the explainable AI (71% accuracy) and the black-box AI (77% accuracy) reveals a trade-off between interpretability and performance—organizations must weigh whether the improved human understanding and trust from explanations justifies accepting lower technical accuracy in clinical applications.