An Empirical Study of Modular Bias Mitigators and Ensembles

Bias mitigators alone produce unstable results: The paper demonstrates that existing bias mitigation techniques often fail to consistently reduce algorithmic bias when tested across different data splits, highlighting a critical reliability problem in current fairness approaches.

Ensemble methods can improve fairness stability: By combining multiple bias mitigators through ensemble techniques (bagging, boosting, voting, and stacking), the researchers found that fairness outcomes become more stable and predictable—similar to how ensembles have long improved prediction accuracy.

Modular combination of tools is practical: The authors created an open-source library that allows practitioners to flexibly combine 10 different bias mitigators with 4 ensemble methods, making it easier for organizations to experiment with fairness solutions without building custom infrastructure.

Empirical testing across diverse datasets matters: Testing on 13 datasets—including both standard fairness benchmarks and newly curated datasets—revealed that the effectiveness of different mitigator-ensemble combinations varies, emphasizing that one-size-fits-all fairness solutions don't exist.

Actionable guidance for practitioners: The paper distills complex findings into a practical guidance diagram, helping non-experts navigate which combinations of mitigators and ensembles work best for their specific fairness challenges rather than requiring deep technical expertise.