Empirical Likelihood-Based Fairness Auditing: Distribution-Free Certification and Flagging

More reliable bias detection without assumptions: The paper introduces a statistical method that can audit AI systems for fairness without requiring assumptions about how data is distributed, making it applicable to real-world datasets where underlying distributions are often unknown or complex.

Faster auditing at scale: The proposed empirical likelihood approach is "several orders of magnitude" faster than existing bootstrap-based methods while maintaining accuracy, which is critical for organizations needing to regularly audit large-scale AI systems used in high-stakes decisions like recidivism prediction and hiring.

Catches intersectional biases: The framework successfully identified intersectional biases in the COMPAS recidivism prediction dataset (a widely-studied case of algorithmic bias), demonstrating it can detect discrimination affecting specific demographic combinations, not just single demographic groups.

Two complementary auditing functions: The method performs both certification (verifying a model meets fairness standards) and flagging (identifying which demographic groups are experiencing disparate treatment), providing organizations with both compliance verification and actionable insights about where bias occurs.

Statistically valid without restrictive assumptions: Unlike traditional auditing techniques, the approach produces results with valid statistical guarantees (asymptotically chi-square distributions) without imposing restrictive assumptions about data, reducing the risk of false conclusions about fairness.