Collective eXplainable AI: Explaining Cooperative Strategies and Agent Contribution in Multiagent Reinforcement Learning with Shapley Values

Shapley values can explain how AI agents contribute to team decisions: The paper demonstrates that Shapley values—a concept from game theory—successfully measure each agent's contribution in cooperative multi-agent reinforcement learning systems, providing transparency into how AI teams make decisions together.

This approach addresses a critical gap in AI transparency: While explainability methods exist for many AI applications, deep reinforcement learning remains poorly understood. Since RL is increasingly used in critical public applications, this work provides a needed tool for making these systems more interpretable and trustworthy.

The method has practical limitations for real-time accountability: The paper acknowledges that Shapley values provide only general explanations about overall model behavior and cannot explain individual actions or specific episodes. This means the approach is better suited for auditing AI systems after deployment rather than justifying particular decisions in real-time.

Potential applications extend beyond games to fairness and policy: The authors suggest implications for non-discriminatory decision-making, ethical AI systems, and policy-making under fairness constraints—indicating this technique could help ensure AI systems treat different stakeholders equitably in real-world scenarios.

Computational efficiency remains a challenge: The paper addresses the high computational overhead of Shapley values through Monte Carlo sampling approximation, but this limitation may still affect practical deployment in resource-constrained environments.