ExplainabilityTransparencyTrustworthinesscs.LG
The future of human-centric eXplainable Artificial Intelligence (XAI) is not post-hoc explanations
Vinitra Swamy, Jibril Frej, Tanja Käser
July 1, 2023
Journal of Artificial Intelligence Research
35 citations
1 influential
Key Takeaways
-
Post-hoc explanations are unreliable: Current AI explanation methods that try to explain already-built "black-box" models systematically disagree with each other when analyzing the same decisions, meaning we cannot trust a single explanation tool to tell us why an AI made a particular choice.
-
We need explanations we can act on: The paper identifies five critical requirements for trustworthy AI in real-world applications like healthcare and education: explanations must be available in real-time, accurate, actionable (useful for decision-making), understandable to humans, and consistent across different explanation methods.
-
Building interpretability in from the start is better than explaining later: Rather than trying to reverse-engineer why a complex AI system made a decision after the fact, the authors argue AI systems should be designed from the ground up to be interpretable—making their reasoning transparent as part of their core architecture.
-
The middle path is the solution: The paper rejects both extremes—neither should we rely on unexplainable AI systems with added explanations, nor should we abandon modern neural networks entirely. Instead, we should develop new types of neural networks that are inherently interpretable by design.
-
This matters for high-stakes decisions: As AI systems become larger and more embedded in critical areas like healthcare, education, and personalized services, the inability to reliably explain their decisions poses real risks that current XAI approaches cannot adequately address.