ExplainabilityTransparencyAccountabilitycs.HC
Explainable AI, but explainable to whom?
Julie Gerlings, Millie Søndergaard Jensen, Arisa Shollo
June 10, 2021
Handbook of Artificial Intelligence in Healthcare
61 citations
3 influential
Key Takeaways
-
Different stakeholders need different explanations: The research reveals that explainability isn't one-size-fits-all. In their COVID-19 ICU classification system, the development team, subject matter experts, decision makers, and end users each had distinct explanation needs—meaning a single explanation won't satisfy everyone involved in an AI system.
-
"Explainable to whom?" is the critical question: The paper challenges the assumption that there is just one "user" who needs explanations. By identifying multiple stakeholder groups with different concerns, the authors highlight that responsible AI implementation requires asking who specifically needs to understand the system and why, rather than treating explainability as a generic requirement.
-
Healthcare context demands stakeholder-specific approaches: The study demonstrates that in high-stakes healthcare settings, AI systems must be adjusted to support different stakeholders' needs. This practical insight suggests that successful and responsible deployment of AI in healthcare requires understanding the explanation requirements of clinicians, administrators, developers, and other relevant parties separately.
-
Explainability addresses multiple concerns simultaneously: The research shows that the need for explainable AI emerges from different concerns depending on the stakeholder group—whether related to trust, transparency, validation, or decision-making. This suggests that responsible AI governance must map which stakeholders have which concerns before designing explanations.