Towards Quantification of Explainability in Explainable Artificial Intelligence Methods

There is currently no agreed-upon standard for measuring explainability in AI systems, despite their widespread use in critical domains like healthcare, finance, and criminal justice. This paper addresses this gap by proposing the first model-agnostic framework to quantify explainability, filling an important void in responsible AI practice.

Explainability requires input from multiple disciplines, including psychology and social science, not just computer science. The paper emphasizes that truly human-friendly explanations must account for how people actually understand and trust information, making explainability a multidisciplinary challenge rather than a purely technical one.

The ability to measure explainability is essential for high-stakes decisions affecting human rights and lives. By developing quantifiable metrics for explainability, organizations can now objectively assess whether their AI systems provide adequate transparency in sensitive domains like criminal justice and medicine, rather than relying on subjective claims.

A "model-agnostic" approach to quantification means the framework can work across different types of AI systems, making it broadly applicable. This is significant because it allows organizations using various AI technologies to use a common standard for evaluating and comparing the explainability of their systems.