Model Mis-specification and Algorithmic Bias

Clean data doesn't guarantee fair outcomes: The paper demonstrates that even when input data contains no bias, algorithmic bias can still emerge if the model is incorrectly specified—meaning the mathematical model chosen doesn't accurately match the real-world relationships in the data.

Overall accuracy masks group-level disparities: A critical finding is that an algorithm can show negligible average prediction errors across an entire population while simultaneously producing large, unequal errors for different demographic groups—a "hidden bias" problem that standard performance metrics may miss.

Bias can manifest with opposite signs across groups: In the worst case, the paper shows that two equally-sized demographic groups can experience prediction errors of equal magnitude but opposite directions, meaning the algorithm systematically over-predicts for one group while under-predicting for the other.

Model choice matters as much as data quality: The research indicates that bias stems not just from problematic training data but fundamentally from how well the chosen model captures the true relationships between features and outcomes—suggesting that responsible AI requires careful attention to model selection, not just data curation.

Mathematical properties of data drive bias patterns: The paper provides closed-form mathematical relationships showing that bias is directly determined by the statistical properties (first and second moments) of the features in the data, offering a foundation for understanding and potentially predicting where algorithmic bias will emerge.