AccountabilityExplainabilityTransparencycs.AI
Actual Causality and Responsibility Attribution in Decentralized Partially Observable Markov Decision Processes
Stelios Triantafyllou, Adish Singla, Goran Radanovic
April 1, 2022
AAAI/ACM Conference on AI, Ethics, and Society
19 citations
1 influential
Key Takeaways
-
Actual causality in multi-agent systems requires new frameworks: The paper establishes that existing causality definitions from single-agent settings can produce counter-intuitive results in decentralized multi-agent decision-making scenarios, necessitating a novel definition that better accounts for how agents' actions causally depend on each other.
-
Responsibility goes beyond identifying causes: The research argues that fairly attributing responsibility to an agent requires considering not just whether they caused an outcome, but also how many causal pathways they participated in and whether they had the ability to control their own degree of responsibility—suggesting simple "who caused this?" analyses are insufficient for accountability.
-
Technical bridge between AI decision-making and causality: By connecting decentralized partially observable Markov decision processes (the mathematical framework used in multi-agent AI systems) to structural causal models (a formal language for causality), the paper provides tools for rigorously analyzing accountability in complex AI systems where multiple agents make decisions with incomplete information.
-
Implications for AI accountability systems: The framework enables more precise identification of which specific decisions by which agents were actually critical to outcomes, rather than attributing responsibility based on correlation or proximity to an event—important for auditing and explaining AI system behavior in real-world applications.