ExplainabilityAccountabilityTrustworthinesscs.CL
Process Knowledge-infused Learning for Clinician-friendly Explanations
Kaushik Roy, Yuxin Zi, Manas Gaur, Jinendra Malekar, Qi Zhang, Vignesh Narayanan, Amit Sheth
June 16, 2023
AAAI Summer Symposia
19 citations
Key Takeaways
-
Clinical alignment improves trust in AI explanations: The paper demonstrates that when AI explanations are structured around how clinicians actually think about diagnoses, clinicians find them significantly more useful—PK-iL achieved 70% agreement with clinicians compared to only 47% for standard explainability methods, suggesting that technical explanations alone are insufficient for healthcare applications.
-
Social media mental health screening requires clinical validation: The research identifies a critical gap in current language model approaches: they analyze social media posts for mental health signals but don't validate their reasoning against actual clinical diagnostic processes, highlighting the importance of grounding AI health tools in established medical practice rather than purely statistical patterns.
-
Explainability methods need domain-specific design: The study shows that generic explainability techniques fail in specialized contexts like mental health assessment, with standard XAI methods achieving only 47% clinician agreement. This suggests that responsible AI deployment in healthcare requires customized explanation approaches rather than one-size-fits-all solutions.
-
New benchmark for suicidality assessment addresses a critical gap: By releasing a new dataset for assessing suicidality with clinical process knowledge annotations, the paper provides infrastructure for future research that bridges AI capabilities and clinical practice—an important step toward responsible development of mental health screening tools.
-
Process knowledge structures are key to interpretability: The core innovation—layering clinical diagnostic frameworks onto language model outputs—demonstrates that responsible AI in healthcare requires embedding domain expertise into model design, not just adding explanations after the fact.