Explaining RL Decisions with Trajectories

RL systems need better explanations for real-world use: The paper identifies explanation as "a key component for the adoption of reinforcement learning in many real-world decision-making problems," highlighting that current approaches focusing only on individual features may be insufficient for understanding AI decisions in critical applications.

Training data matters more than we typically explain: Rather than just explaining what features an RL agent is looking at, this work proposes tracing decisions back to specific examples the agent learned from during training. This trajectory-based approach helps answer "why did the AI decide this?" by showing which past experiences influenced the decision.

The method works across different types of problems: The researchers demonstrated their approach on diverse environments including grid-worlds, video games (Atari), and robotic control tasks with both simple and complex decision spaces, suggesting the explanation method could be broadly applicable to different RL applications.

Human understanding aligns with data-driven explanations: A human study showed that people's understanding of an RL agent's behavior matched well with the explanations generated by attributing decisions to training trajectories, suggesting this approach produces explanations that are intuitive and meaningful to humans—an important validation for responsible AI deployment.

Offline RL explanations are particularly important: The focus on offline RL (where agents learn from fixed datasets rather than live interaction) is significant because many real-world applications use this approach, making explainability especially critical for high-stakes domains like healthcare or autonomous systems.