Demystifying Reinforcement Learning in Production Scheduling via Explainable AI

Current XAI methods fall short of real-world needs: The paper finds that popular explainability techniques like SHAP and Captum "lack falsifiability and consistent terminology" and "do not adequately consider domain-knowledge, the target audience or real-world scenarios," meaning organizations cannot reliably verify whether AI explanations are actually correct.

AI explanations need verification, not just description: Rather than simply showing which inputs influenced a scheduling decision, the authors propose a "hypotheses-based workflow" that tests whether explanations align with actual domain knowledge and the AI agent's underlying objectives—emphasizing that explanations must be verified before being trusted.

One-size-fits-all explanations don't work: The paper demonstrates that effective AI transparency requires "tailoring hypotheses to the target audience," suggesting that explaining AI decisions to engineers, managers, or operators requires different approaches rather than a single technical explanation.

Responsible AI deployment requires domain expertise: The study shows that understanding why a DRL agent makes scheduling decisions demands integrating explainability methods with real-world operational knowledge, indicating that responsible AI implementation cannot rely on technical tools alone but must involve subject matter experts.

Causal understanding matters more than input correlations: The authors note that current methods "typically provide simple input-output explanations rather than causal interpretations," highlighting that knowing which factors influenced a decision is less useful than understanding why those factors matter for the business outcome.