Who's responsible? Jointly quantifying the contribution of the learning algorithm and training data

Both algorithms and datasets share responsibility for AI failures: When an AI system performs poorly on certain groups, the paper shows that blame cannot be assigned to just one source. Using a framework called Extended Shapley, researchers can quantify how much a specific algorithm versus the training data contributed to the problem—for example, determining whether poor performance on a subpopulation stems from biased data or a flawed algorithm.

Responsibility assessment requires systematic comparison: The paper's approach involves testing counterfactuals: Would performance improve with better data but the same algorithm? Or with a different algorithm but the same data? This rigorous method helps organizations understand whether they should focus on data collection improvements or algorithmic changes when failures occur.

This framework addresses real-world accountability needs: As the authors note, these questions are "far from hypothetical" as ML systems become more widespread. The work provides a practical tool for organizations to answer concrete questions about where problems originate when AI systems fail on specific populations, which is increasingly important for responsible deployment.

The Extended Shapley framework offers a "principled" solution: Rather than relying on intuition or blame-shifting, the paper proposes a mathematically rigorous framework for joint credit assignment, suggesting that responsibility can be measured and quantified in a consistent, defensible way across different AI systems and datasets.