Biased Programmers? Or Biased Data? A Field Experiment in Operationalizing AI Ethics

Biased training data is the primary culprit: The study found that biased algorithmic predictions stem mostly from biased training data rather than from programmer bias, suggesting that data quality should be a primary focus for organizations seeking to reduce algorithmic discrimination.

Better data motivates better work: Interestingly, one-third of the bias reduction benefits from improved training data came through an indirect mechanism—engineers simply worked harder and were more responsive to performance incentives when given higher-quality data, demonstrating that data quality affects not just algorithms but human effort and motivation.

Demographic diversity of programmers alone doesn't guarantee fairness: The research found no evidence that female engineers, minority engineers, or those with lower implicit bias scores produced less biased algorithms, challenging the assumption that simply diversifying engineering teams will automatically reduce algorithmic bias.

Cross-demographic collaboration can improve outcomes: While individual demographic characteristics didn't predict lower bias, the study found that prediction errors varied systematically across demographic groups, suggesting that averaging predictions across demographically diverse teams could reduce overall bias.

Simple interventions have measurable but limited effects: Technical advice, reminders about bias, and improved incentive structures all showed benefits for reducing algorithmic bias, but the study quantifies these as modest compared to the impact of using better training data, helping organizations prioritize their bias-reduction efforts.