ExplainabilityTransparencyTrustworthinesscs.HC
Explainable AI does not provide the explanations end-users are asking for
Savio Rozario, George Čevora
January 25, 2023
arXiv.org
4 citations
Key Takeaways
-
XAI tools often fail to meet user needs: The paper finds that explainable AI techniques frequently don't provide the types of explanations that end-users actually want, suggesting a significant gap between what XAI developers create and what stakeholders require to understand AI systems.
-
XAI has unintended consequences when deployed: While XAI methods may be useful during development and testing phases, their real-world adoption by organizations can backfire, potentially creating false confidence or misleading users rather than genuinely enhancing trust.
-
Transparency and validation are more effective than XAI alone: The authors argue that rigorous validation of AI systems combined with transparent documentation about how they work and their limitations provides a better foundation for trustworthy AI than relying on XAI explanations.
-
Trust requires more than technical explanations: The research suggests that building genuine trust in AI systems involves more than making models technically explainable—it requires comprehensive approaches that address user concerns and organizational accountability.
Abstract
Explainable Artificial Intelligence (XAI) techniques are frequently required by users in many AI systems with the goal of understanding complex models, their associated predictions, and gaining trust. While suitable for some specific tasks during development, their adoption by organisations to enhance trust in machine learning systems has unintended consequences. In this paper we discuss XAI's limitations in deployment and conclude that transparency alongside with rigorous validation are better suited to gaining trust in AI systems.