An Objective Metric for Explainable AI: How and Why to Estimate the Degree of Explainability

Explainability needs objective measurement: The paper addresses a critical gap in responsible AI by proposing the first model-agnostic metric to objectively measure how explainable an AI system actually is, rather than relying on subjective assessments or assumptions about what makes something "explainable."

The metric was validated with real users: The researchers tested their approach with over 190 participants across realistic healthcare and finance applications, finding statistically significant results (P < .01), which suggests the metric reliably captures what people actually find explainable rather than just theoretical ideals.

Different AI systems have different explainability levels: By testing both neural networks and TreeSHAP (a popular explanation tool), the study demonstrates that explainability varies across different AI technologies—meaning organizations cannot assume one explanation method works equally well for all systems.

Philosophy-based approach grounds explainability in theory: Rather than inventing a new definition, the researchers grounded their metric in Achinstein's Theory of Explanations from philosophy, providing a principled foundation for what counts as a genuine explanation and making the metric more defensible for regulatory or compliance purposes.

Practical tool for responsible AI governance: This metric could help organizations systematically evaluate whether their AI systems meet explainability requirements, moving beyond marketing claims to concrete, measurable standards for transparency and accountability.