BiasFairnessAccountabilitycs.LG
Target specification bias, counterfactual prediction, and algorithmic fairness in healthcare
Eran Tal
August 3, 2023
AAAI/ACM Conference on AI, Ethics, and Society
24 citations
Key Takeaways
-
Target specification bias is a distinct problem from data bias: The paper identifies that ML bias in healthcare stems not just from unrepresentative data or health disparities, but from a fundamental mismatch between what the model predicts and what clinicians actually need to know—a problem that "persists independently of data limitations and health disparities."
-
The mismatch involves counterfactual vs. actual outcomes: Decision makers typically want to predict what would happen under different treatment scenarios (counterfactual predictions), but ML models often predict what actually happened in historical data, creating a subtle but consequential misalignment in target variables.
-
Uncorrected target specification bias has real clinical harms: When left unaddressed, this bias leads to overestimated model accuracy, inefficient use of medical resources, and "suboptimal decisions that can harm patients"—making it a patient safety issue, not just a technical one.
-
Measurement science offers solutions: The paper suggests that principles from metrology (the science of measurement) can help counteract target specification bias, pointing toward practical approaches for ensuring ML tools actually measure what clinicians need them to measure.
Abstract
Bias in applications of machine learning (ML) to healthcare is usually attributed to unrepresentative or incomplete data, or to underlying health disparities. This article identifies a more pervasive source of bias that affects the clinical utility of ML-enabled prediction tools: target specification bias. Target specification bias arises when the operationalization of the target variable does not match its definition by decision makers. The mismatch is often subtle, and stems from the fact that decision makers are typically interested in predicting the outcomes of counterfactual, rather than actual, healthcare scenarios. Target specification bias persists independently of data limitations and health disparities. When left uncorrected, it gives rise to an overestimation of predictive accuracy, to inefficient utilization of medical resources, and to suboptimal decisions that can harm patients. Recent work in metrology - the science of measurement - suggests ways of counteracting target specification bias and avoiding its harmful consequences.