Training Characteristic Functions with Reinforcement Learning: XAI-methods play Connect Four

Existing XAI methods rely on unrealistic scenarios: Current explanation methods for AI systems often evaluate what happens when inputs are artificially altered in ways that don't reflect real-world data (called "off-manifold" inputs), making it difficult to trust whether their explanations are actually meaningful.

Game-based testing provides a clearer benchmark: By training AI agents to play Connect Four with randomly hidden information, the researchers created a controlled environment where it's objectively clear which input features (board positions) should matter for decisions, allowing fair comparison of different explanation methods.

Characteristic functions offer a theoretically sound foundation: The paper grounds its approach in cooperative game theory's characteristic functions, which provide a mathematically principled way to determine which input components contributed to a decision—moving beyond ad-hoc explanation methods toward more rigorous fairness guarantees.

The ambiguity problem in current XAI is significant: Most black-box explanation methods (like those for neural networks) lack a clear, agreed-upon way to represent "partial inputs," meaning different methods may be answering slightly different questions when they claim to explain the same decision.

Practical validation through competition is possible: Rather than relying solely on theoretical arguments, the researchers' approach allows different XAI methods to be directly compared by having their explanations tested in actual gameplay, providing empirical evidence about which methods identify truly relevant features.