Error Parity Fairness: Testing for Group Fairness in Regression Tasks

Fairness testing for regression tasks has been overlooked: While most AI fairness research focuses on classification (yes/no decisions), this paper addresses a gap by developing fairness testing methods for regression tasks—AI systems that make numerical predictions rather than categorical decisions, which are common in healthcare, finance, and resource allocation.

Error parity provides a measurable fairness standard: The paper proposes "error parity" as a fairness metric for regression, checking whether prediction errors are distributed similarly across sensitive groups (e.g., different races or genders). If one group consistently experiences larger prediction errors, this indicates potential unfairness in the system.

Statistical testing can identify which groups are disadvantaged: The methodology uses permutation tests to compare prediction errors across groups, allowing auditors to detect which specific populations are being treated unfairly by an AI system and by how much—moving beyond simply knowing unfairness exists.

Real-world case study reveals racial disparities in COVID-19 forecasts: When applied to COVID-19 projections at the US county level, the testing methodology revealed "race-based differences in forecast errors," demonstrating that this fairness testing approach can uncover concrete disparities in deployed AI systems affecting public health decisions.

This work supports AI accountability and algorithm audits: By providing a practical testing framework, this research enables organizations to conduct fairness audits of regression-based AI systems as part of broader accountability assessments, helping meet legal and social expectations for responsible AI deployment.