Defining Explainable AI for Requirements Analysis

Explainability requirements vary by application: The paper identifies that different AI applications need different types of explanations to build trust. Rather than a one-size-fits-all approach, organizations must first understand what their specific use case requires before selecting an AI system.

Three key dimensions for defining explainability needs: The authors propose categorizing explanation requirements along three dimensions—Source (where the explanation comes from), Depth (how detailed it is), and Scope (how broad it is)—providing a framework for organizations to articulate what they actually need from their AI systems.

Matching requirements to technical capabilities is critical: The paper emphasizes the importance of aligning what an application requires in terms of explanations with what the underlying machine learning techniques can actually deliver, suggesting that many implementations may fail because this match is never properly established.

Trust requires more than just good performance: The research reinforces that AI systems earning user trust depends not only on making accurate decisions but on being able to explain why those decisions were made and demonstrating they're based on appropriate reasoning—a distinction that has significant implications for how organizations should evaluate and deploy AI.