Unlocking the Black Box: Analysing the EU Artificial Intelligence Act's Framework for Explainability in AI

The EU AI Act recognizes explainability as fundamental, but implementation details remain unclear. While the regulation establishes explainability as a core principle for responsible AI, the paper notes that "the exact XAI techniques and requirements are still to be determined and tested in practice," meaning organizations and regulators are still figuring out how to actually implement these requirements.

Explainability is most critical in high-stakes decisions affecting people's lives. The paper identifies healthcare, credit scoring, policing, and criminal justice as fields where understanding how AI makes decisions is essential for accountability, ethics, and fairness—areas where a wrong decision can significantly harm individuals.

There's a gap between XAI techniques and real-world governance. The paper highlights "challenges of implementing the principle of explainability in AI governance and policies," suggesting that while technical solutions for making AI more transparent exist, translating these into enforceable regulations and practical oversight mechanisms remains difficult.

Standard-setting, oversight, and enforcement are major unresolved issues. The paper emphasizes that integrating XAI into EU law requires addressing three critical gaps: establishing what standards should apply, who oversees compliance, and how violations will be enforced—none of which have clear answers yet.

Explainability is essential for mitigating AI risks. The paper identifies the "lack of explainability of Artificial Intelligence" as "one of the first obstacles that the industry and regulators must overcome to mitigate the risks associated with the technology," positioning transparency as foundational to responsible AI deployment.