Synthesising Reinforcement Learning Policies through Set-Valued Inductive Rule Learning

Black-box RL policies can be converted into human-readable rules: The paper demonstrates that complex neural network-based reinforcement learning policies can be distilled into a small set of interpretable decision rules, making it possible for non-experts to understand what an AI agent has learned without requiring technical knowledge of deep learning.

Action value information improves explainability: By leveraging meta-information about near-optimal alternative actions (not just the single best action), the researchers reduced the number of rules needed to explain policies, suggesting that understanding why actions are equally good can make AI systems more transparent than simply knowing the top choice.

Interpretability doesn't require sacrificing performance: The paper includes a refinement algorithm that fine-tunes extracted rules to maintain good performance in the actual environment, addressing a key concern that simplified explanations might come at the cost of effectiveness—demonstrating that understandable AI and capable AI aren't necessarily in conflict.

Complex real-world tasks can be explained simply: The approach successfully distills policies from the Mario AI benchmark, a complex task requiring modern neural networks, into just a few rules, suggesting this method could scale to other challenging domains where stakeholders need to understand AI decision-making.

This approach supports trustworthiness and accountability: By making RL policies interpretable, this work enables humans to verify that learned behaviors are appropriate, identify potential biases, and take responsibility for AI system decisions—critical requirements for deploying AI in high-stakes applications.