A Critical Review of Predominant Bias in Neural Networks

Two distinct types of bias require different solutions: The paper identifies a critical confusion in AI research between performance disparity across demographic groups and algorithmic reliance on protected attributes (like race or gender). These are fundamentally different problems requiring separate debiasing approaches, yet the field has conflated them, leading to ineffective solutions.

The confusion is widespread and consequential: After reviewing numerous papers, the authors found that this conceptual mix-up has "significantly hampered the clarity of the community and subsequent development of debiasing methodologies," meaning flawed bias mitigation efforts may be deployed in real-world systems without addressing the actual problems.

Different biases have different root causes and require different fixes: The paper's experiments across synthetic, census, and image datasets demonstrate that performance disparities and reliance on protected attributes stem from distinct underlying causes and cannot be solved with one-size-fits-all debiasing methods—organizations need to diagnose which type of bias they're addressing first.

Current bias assessment metrics are inconsistently effective: The research evaluates multiple bias metrics and finds they vary significantly in their ability to properly assess whether different types of bias have actually been mitigated, suggesting that companies using standard fairness metrics may have false confidence in their AI systems' fairness.

Clarity on bias definitions is foundational for trustworthy AI: By providing mathematical definitions and unifying the literature, this work establishes a common language necessary for the field to develop more rigorous, effective approaches to building fair and trustworthy AI systems.