The Contribution of XAI for the Safe Development and Certification of AI: An Expert-Based Analysis

XAI methods can help identify problems during development: According to the expert interviews, explainability techniques are useful for revealing biases and failures in machine learning models during the development phase, making them a valuable tool for improving AI safety before deployment.

Certification faces fundamental limitations with XAI alone: The study finds that while XAI methods provide helpful insights, their impact on formal AI certification is expected to be "limited" because certification requires "comprehensive and correct information about technical systems"—a standard that current explainability methods struggle to consistently meet.

The black-box problem persists despite XAI advances: The paper identifies that the inherent opacity of machine learning models remains a significant barrier to using conventional certification approaches, and XAI methods, while promising, do not fully resolve this fundamental challenge.

Regulatory compliance requires more than explanation: With regulations like the EU AI Act creating new requirements for trustworthy AI, the research suggests that explainability alone is insufficient for meeting certification standards—additional safeguards and verification methods are needed alongside XAI.

Expert consensus acknowledges both promise and gaps: The qualitative analysis of 15 experts from AI and certification fields reveals a nuanced view: XAI is recognized as "a helpful asset" for safe AI development, but stakeholders acknowledge significant shortcomings in using these methods as the primary basis for certifying complex AI systems.