A Scoresheet for Explainable AI

Existing standards for AI transparency are too vague to be useful in practice. The paper identifies a critical gap between high-level regulatory requirements for transparency and the specific, actionable explainability standards that developers and organizations actually need to implement.

Different stakeholders need different types of explanations. The scoresheet was developed by considering requirements across multiple stakeholder groups, recognizing that a regulator, end-user, and system developer may each need distinct information to understand how an AI system works.

A practical assessment tool can help organizations evaluate their explainability efforts. Rather than relying on subjective judgments, the scoresheet provides a structured framework that organizations can use to either set explainability requirements upfront or assess how well their existing AI systems meet explainability standards.

The framework applies broadly across AI systems. The paper demonstrates that the scoresheet's approach is generalizable beyond specific AI technologies, including multiagent systems and other autonomous systems, suggesting it could serve as a common standard across different AI applications.

Explainability supports appropriate trust in AI systems. The paper emphasizes that transparency through explainability helps users develop calibrated trust—neither over-relying on nor unfairly distrusting AI systems—which is essential for responsible AI deployment.