Contrastive Explanations for Reinforcement Learning in terms of Expected Consequences

RL agents need better explanations: While machine learning transparency has advanced for classification models, reinforcement learning systems—which make sequential decisions—lag behind in explainability, creating barriers to trust and the ability to correct these systems when they make mistakes.

Consequences matter more than individual decisions: The pilot survey found that users prefer explanations focused on an agent's overall policy and its expected outcomes rather than explanations of single actions, suggesting people want to understand the "big picture" impact of AI decisions.

Contrastive explanations are more useful: The paper proposes comparing what an RL agent actually does against alternative policies users might ask about, helping people understand not just what the agent does, but why it chose that approach over other options.

Translation to human language is essential: The method requires converting technical states and actions into descriptions humans can understand, highlighting that explainability requires bridging the gap between how AI systems represent information and how people naturally think about decisions and their consequences.

Practical implementation is feasible: The researchers developed a working procedure to generate these explanations, suggesting that making RL systems more transparent and trustworthy is not just theoretically possible but can be implemented in practice.