ExplainabilityTransparencyAccountabilitycs.AI
Explainable Artificial Intelligence and Machine Learning: A reality rooted perspective
Frank Emmert-Streib, Olli Yli-Harja, Matthias Dehmer
January 26, 2020
WIREs Data Mining Knowl. Discov.
100 citations
4 influential
Key Takeaways
-
Black box AI models pose real risks in high-stakes domains: The paper identifies health sciences and criminal justice as fields where the lack of transparency in AI systems—particularly deep learning methods—creates "severe problems," highlighting that explainability isn't merely a nice-to-have feature but a necessity for responsible deployment.
-
Explainability has practical limits rooted in scientific reality: Rather than presenting an idealized vision of what explainable AI "should be," the paper emphasizes what it "can be" based on grounded scientific theory, suggesting that stakeholders should have realistic expectations about the transparency achievable in complex AI systems rather than assuming all black boxes can be fully opened.
-
The explainability challenge extends beyond just technology: The paper frames explainability as one of "vast challenges" in analyzing big data across nearly all scientific fields, indicating that this is a systemic issue requiring broader consideration of how we validate and trust AI systems across multiple domains, not just isolated technical fixes.
Abstract
We are used to the availability of big data generated in nearly all fields of science as a consequence of technological progress. However, the analysis of such data possess vast challenges. One of these relates to the explainability of artificial intelligence (AI) or machine learning methods. Currently, many of such methods are non-transparent with respect to their working mechanism and for this reason are called black box models, most notably deep learning methods. However, it has been realized that this constitutes severe problems for a number of fields including the health sciences and criminal justice and arguments have been brought forward in favor of an explainable AI. In this paper, we do not assume the usual perspective presenting explainable AI as it should be, but rather we provide a discussion what explainable AI can be. The difference is that we do not present wishful thinking but reality grounded properties in relation to a scientific theory beyond physics.