AI2030 Responsible AI Tools Marketplace

Key Takeaways

Explainability requirements in the EU AI Act lack concrete technical specifications, forcing organizations to interpret vague legal language about what "explainability" actually means in practice—creating uncertainty about whether current XAI methods satisfy regulatory obligations.
Current XAI research prioritizes technical metrics over real-world regulatory compliance, meaning most academic explainability work may not directly address what regulators and organizations actually need to demonstrate legal compliance for high-risk AI systems.
End-users and affected people are largely absent from both XAI research and AI Act regulatory frameworks, even though they are the stakeholders most impacted by AI decisions—creating a gap where explainability methods focus on provider-deployer relationships rather than helping individuals understand decisions affecting them.
Human-centered XAI practitioners have an opportunity to actively shape how regulations get implemented by defining their own roles in compliance and making explicit the connection between end-users and affected parties, rather than passively waiting for regulators to specify technical requirements.
Bridging the gap between legal requirements and technical solutions requires interdisciplinary collaboration between legal experts, engineers, and requirements specialists—a real-world clinical case study demonstrated that alignment between XAI methods and regulatory obligations is possible but requires deliberate cross-disciplinary effort.

Abstract

The EU AI Act makes explainability urgent for high-risk AI systems, yet most XAI research focuses on technical metrics rather than regulatory compliance. Understanding how legal requirements reshape XAI method design is challenging: the AI Act regulates organizational relationships (providers, deployers) using legal terminology, specifies obligations without concrete technical requirements, and underrepresents end-users--the very stakeholders whose needs human-centered XAI addresses. As regulations emerge globally, human-centered XAI practitioners face both a challenge and an opportunity: regulations pull XAI research toward real-world deployment, while practitioners can actively shape how explainability enables compliance. This establishes a bidirectional relationship. Our contribution is threefold. First, we provide the first interdisciplinary analysis of XAI's role in the AI Act--conducted by a team comprising AI Act legal experts, ML engineers, and requirements engineers--on a real-world clinical decision support system. Second, we systematically align XAI stakeholder roles with AI Act legal responsibilities, revealing where explainability methods address regulatory requirements versus where additional measures are necessary. Third, we identify three key opportunities for human-centered XAI practitioners: actively defining their roles in regulatory implementation; making the user-to-affected-party relationship explicit where regulations address only provider-deployer obligations; and enabling compliance while building multi-level trust--from regulators to affected parties.

Paper accepted at the Fourth World Conference on Explainable Artificial Intelligence, xAI 2026, Fortaleza, Brazil, July 1-3, 2026

Key Takeaways

Explainability requirements in the EU AI Act lack concrete technical specifications, forcing organizations to interpret vague legal language about what "explainability" actually means in practice—creating uncertainty about whether current XAI methods satisfy regulatory obligations.
Current XAI research prioritizes technical metrics over real-world regulatory compliance, meaning most academic explainability work may not directly address what regulators and organizations actually need to demonstrate legal compliance for high-risk AI systems.
End-users and affected people are largely absent from both XAI research and AI Act regulatory frameworks, even though they are the stakeholders most impacted by AI decisions—creating a gap where explainability methods focus on provider-deployer relationships rather than helping individuals understand decisions affecting them.
Human-centered XAI practitioners have an opportunity to actively shape how regulations get implemented by defining their own roles in compliance and making explicit the connection between end-users and affected parties, rather than passively waiting for regulators to specify technical requirements.
Bridging the gap between legal requirements and technical solutions requires interdisciplinary collaboration between legal experts, engineers, and requirements specialists—a real-world clinical case study demonstrated that alignment between XAI methods and regulatory obligations is possible but requires deliberate cross-disciplinary effort.

Abstract

Paper accepted at the Fourth World Conference on Explainable Artificial Intelligence, xAI 2026, Fortaleza, Brazil, July 1-3, 2026

The Bidirectional Relationship Between XAI and Regulation: Operationalizing XAI for the AI Act

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links

The Bidirectional Relationship Between XAI and Regulation: Operationalizing XAI for the AI Act

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links