Audits Under Resource, Data, and Access Constraints: Scaling Laws For Less Discriminatory Alternatives

The legal burden of proving discrimination in AI cases falls on claimants who often lack resources to train competing models. The paper identifies a critical asymmetry: anti-discrimination law requires proving a "less discriminatory alternative" exists, but claimants typically have far fewer computational resources and data access than the AI developers they're challenging.

Researchers developed a method allowing discrimination audits with limited compute, data, and model access. The paper presents a procedure that enables claimants to determine whether a less discriminatory AI model is feasible without needing to fully train one, addressing the practical barriers that have made AI discrimination cases difficult to pursue.

A mathematical formula can predict fairness-performance tradeoffs for large models based on small-scale testing. The authors provide a closed-form upper bound for the loss-fairness Pareto frontier that allows claimants to extrapolate what's possible for a contested model by testing at smaller scales, potentially democratizing the ability to audit AI systems.

This approach could level the playing field between individual claimants and well-resourced AI developers. By reducing the expertise, compute power, and data access needed to mount a credible discrimination case, the method addresses a significant practical barrier to AI accountability through legal channels.