Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanations

Explanations in high-stakes domains like medicine are essential for accountability: The paper emphasizes that in domains where decisions significantly impact human welfare, AI systems must provide explanations so that domain experts (such as medical professionals) can understand and take responsibility for AI-assisted decisions, rather than blindly trusting automated results.

Current explainability methods need systematic evaluation: While various explainable AI techniques exist (such as layer-wise relevance propagation that highlights which parts of input data influenced a decision), the paper identifies a critical gap—there is no standardized way to measure whether these explanations are actually good or useful, which is why the authors developed the System Causability Scale (SCS).

Human-AI interaction should enable verification, not just transparency: The paper argues that effective explainability goes beyond simply showing what the AI considered; it should allow human experts to "re-enact and retrace" the AI's reasoning to check whether the explanation makes sense and is plausible, requiring new types of interfaces designed for this interactive process.

Quality of explanations directly impacts responsible AI deployment: By introducing a measurement scale for explanation quality, the paper addresses a practical need for regulators and professionals who must ensure that explainable AI systems actually help humans understand and appropriately oversee AI decision-making in critical applications.