Akshaj Kumar Veldanda, Ivan Brugere, Jiahao Chen, Sanghamitra Dutta, Alan Mishler, Siddharth Garg
Over-parameterized models create a "fairness illusion": When deep neural networks become very large and overfit perfectly to training data (achieving zero training loss), they automatically appear fair on that training data, which causes fairness-correction tools like MinDiff to stop working—even though the underlying bias problem persists on real-world test data.
Popular fairness tools may fail at scale: MinDiff, a fairness method implemented in TensorFlow's Responsible AI Toolkit, effectively reduces bias in smaller models but becomes ineffective as models grow larger, suggesting that practitioners deploying large AI systems may have false confidence in their fairness interventions.
Smaller, constrained models can outperform larger ones: The research found that smaller models trained with fairness constraints sometimes achieved better overall accuracy than larger unconstrained models, challenging the common assumption that "bigger is always better" in machine learning.
Fairness-accuracy tradeoffs are more complex than assumed: The findings suggest that the relationship between model size, fairness, and accuracy is more nuanced than previously understood, and that simply scaling up models while applying fairness corrections is not a reliable strategy for achieving both performance and equity.
Current fairness measures have blind spots: The paper highlights that disparity-based fairness metrics (those measuring errors or accuracy differences) are particularly vulnerable to this problem, though other fairness definitions like demographic parity may not be affected in the same way.
The success of DNNs is driven by the counter-intuitive ability of over-parameterized networks to generalize, even when they perfectly fit the training data. In practice, test error often continues to decrease with increasing over-parameterization, referred to as double descent. This allows practitioners to instantiate large models without having to worry about over-fitting. Despite its benefits, however, prior work has shown that over-parameterization can exacerbate bias against minority subgroups. Several fairness-constrained DNN training methods have been proposed to address this concern. Here, we critically examine MinDiff, a fairness-constrained training procedure implemented within TensorFlow's Responsible AI Toolkit, that aims to achieve Equality of Opportunity. We show that although MinDiff improves fairness for under-parameterized models, it is likely to be ineffective in the over-parameterized regime. This is because an overfit model with zero training loss is trivially group-wise fair on training data, creating an "illusion of fairness," thus turning off the MinDiff optimization (this will apply to any disparity-based measures which care about errors or accuracy. It won't apply to demographic parity). Within specified fairness constraints, under-parameterized MinDiff models can even have lower error compared to their over-parameterized counterparts (despite baseline over-parameterized models having lower error). We further show that MinDiff optimization is very sensitive to choice of batch size in the under-parameterized regime. Thus, fair model training using MinDiff requires time-consuming hyper-parameter searches. Finally, we suggest using previously proposed regularization techniques, viz. L2, early stopping and flooding in conjunction with MinDiff to train fair over-parameterized models.
Citation data from Semantic Scholar