Verifiable Fairness: Privacy-preserving Computation of Fairness for Machine Learning Systems

Fairness audits can now be conducted without exposing sensitive data: The paper introduces a method where fairness metrics are computed on encrypted data (cryptograms), allowing independent auditors, activists, and experts to verify that an ML model is fair without accessing the raw personal information used to train or test it.

Verification is transparent and doesn't require trusting a single authority: Unlike traditional auditing approaches that rely on a trusted third party, FaaS uses zero-knowledge proofs to ensure every computational step is verifiable by the public. This means fairness claims can be independently confirmed without needing to trust any single organization or intermediary.

The approach works across different types of AI systems and fairness definitions: Because FaaS is "model-agnostic and can support various fairness metrics," it can be applied to audit different kinds of machine learning systems (not just one type) and accommodate different definitions of fairness—addressing the reality that fairness means different things in different contexts.

Real-world feasibility has been demonstrated: The authors implemented and tested FaaS on publicly available datasets with thousands of entries, suggesting the approach is practical enough to potentially be deployed as an actual auditing service rather than remaining purely theoretical.

This enables democratic oversight of AI fairness: By making fairness verification publicly accessible and not requiring private channels or proprietary access, the method democratizes the ability to audit AI systems—potentially empowering civil society organizations and affected communities to independently verify fairness claims.