Assessing the Real-World Utility of Explainable AI for Arousal Diagnostics: An Application-Grounded User Study

Explainability matters in clinical practice: When sleep medicine experts used transparent AI (white-box) assistance as a quality-control step, they achieved approximately 30% better performance than with black-box AI, demonstrating that understanding why an AI makes recommendations significantly improves real-world clinical outcomes.

AI-human collaboration reduces disagreement among clinicians: The study found that human-AI teams reduced inter-rater variability compared to unaided experts, suggesting that AI assistance can help standardize clinical decision-making and reduce inconsistency in medical diagnoses.

Timing of AI assistance affects both efficiency and effectiveness: While transparent AI applied at the end (as quality control) improved accuracy by ~30%, AI assistance provided from the start was faster and preferred by most participants, indicating a trade-off between speed and performance that clinicians must navigate.

Trust requires more than accuracy: Despite AI systems matching or exceeding expert performance, clinicians need to understand when and why to trust algorithmic recommendations—suggesting that deploying AI in healthcare requires not just accurate models, but also transparent explanations that enable clinical judgment.

Real-world validation with actual users is essential: Testing with eight professional practitioners revealed practical insights about workflow integration that wouldn't emerge from accuracy metrics alone, highlighting the importance of application-grounded studies before deploying AI in clinical settings.