Deep Learning Reproducibility and Explainable AI (XAI)

Reproducibility is foundational to trustworthy AI: The paper identifies reproducibility as a key component of the EU's Model Governance Framework and emphasizes that it is essential for establishing causality in model interpretation and building public trust in AI systems as they become more widespread.

Deep learning models are inherently unpredictable: The research demonstrates that the nondeterminism (randomness) built into deep learning training algorithms makes it difficult to produce identical results across multiple training runs, which poses a significant challenge for creating reliable and explainable AI systems.

Deterministic AI is technically feasible but challenging: Through experiments with convolutional neural networks for image classification, the authors show that deterministic (fully reproducible) deep learning models are possible to create in practice, though they identify specific limitations and obstacles that must be overcome.

Explainability depends on reproducibility: The paper establishes a direct link between reproducibility and explainability—without being able to reliably reproduce model results, it becomes nearly impossible to explain why an AI system made a particular decision, which undermines accountability and user confidence.

Practical solutions exist but require deliberate effort: The authors provide source code for deterministic models and document both successful approaches and remaining problems, suggesting that achieving reproducible AI requires intentional design choices rather than relying on default deep learning practices.