ExplainabilityTransparencyAccountabilitycs.AI
Dataset | Mindset = Explainable AI | Interpretable AI
Caesar Wu, Rajkumar Buyya, Yuan Fang Li, Pascal Bouvry
August 22, 2024
arXiv.org
Key Takeaways
-
XAI and IAI are distinct concepts, not interchangeable terms: The paper argues that explainable AI (XAI) is actually a subset of interpretable AI (IAI), with XAI focusing on post-hoc analysis of datasets while IAI requires a broader "a priori mindset of abstraction" that goes beyond the data itself.
-
The difference matters for regulated industries: The authors emphasize that distinguishing between XAI and IAI is "indispensable" for determining regulatory policies in high-stakes domains including healthcare, human resources, banking, and finance—suggesting that using these terms interchangeably could lead to inadequate governance frameworks.
-
Interpretability involves both objective and subjective reasoning: While explanations lean toward objective facts about what a model does, interpretations have a subjective dimension—the paper describes this as reasoning that can be directed "outwards" (following laws of nature) or "inwards" (guided by human values and ethics).
-
Trustworthy AI requires understanding both concepts: Both XAI and IAI serve the shared goals of transparency, clarity, fairness, reliability, and accountability in ethical and trustworthy AI systems, but they achieve these goals through different approaches that should be understood separately.
Abstract
We often use "explainable" Artificial Intelligence (XAI)" and "interpretable AI (IAI)" interchangeably when we apply various XAI tools for a given dataset to explain the reasons that underpin machine learning (ML) outputs. However, these notions can sometimes be confusing because interpretation often has a subjective connotation, while explanations lean towards objective facts. We argue that XAI is a subset of IAI. The concept of IAI is beyond the sphere of a dataset. It includes the domain of a mindset. At the core of this ambiguity is the duality of reasons, in which we can reason either outwards or inwards. When directed outwards, we want the reasons to make sense through the laws of nature. When turned inwards, we want the reasons to be happy, guided by the laws of the heart. While XAI and IAI share reason as the common notion for the goal of transparency, clarity, fairness, reliability, and accountability in the context of ethical AI and trustworthy AI (TAI), their differences lie in that XAI emphasizes the post-hoc analysis of a dataset, and IAI requires a priori mindset of abstraction. This hypothesis can be proved by empirical experiments based on an open dataset and harnessed by High-Performance Computing (HPC). The demarcation of XAI and IAI is indispensable because it would be impossible to determine regulatory policies for many AI applications, especially in healthcare, human resources, banking, and finance. We aim to clarify these notions and lay the foundation of XAI, IAI, EAI, and TAI for many practitioners and policymakers in future AI applications and research.