Exploring Bias and Prediction Metrics to Characterise the Fairness of Machine Learning for Equity-Centered Public Health Decision-Making: A Narrative Review

Multiple types of bias can emerge from machine learning systems in public health, not just from the data itself—the review identified various sources of algorithmic bias that can lead to systematic errors in predicting health outcomes for populations, highlighting that bias is a multifaceted problem requiring comprehensive understanding rather than single-point fixes.

Quantitative metrics exist to measure fairness in ML systems, and the review synthesized these measurement approaches from 72 peer-reviewed studies, suggesting that organizations can move beyond subjective assessments and use concrete, measurable standards to evaluate whether their public health algorithms treat different populations equitably.

Current ML applications in public health lack standardized fairness evaluation frameworks—the authors note there is "a lack of comprehensive understanding" of how to systematically assess algorithmic bias in this domain, indicating that many public health institutions may be deploying ML tools without adequate equity-centered evaluation protocols in place.

An equity-centered evaluation framework is now possible, as the review formalizes what types of bias to look for and how to measure them, providing public health decision-makers with a structured approach to ensure ML systems don't inadvertently harm or disadvantage specific populations when used for surveillance, research, or policy decisions.