Mitigating Algorithmic Bias with Limited Annotations

Sensitive data is often unavailable in practice: The paper addresses a real-world constraint where companies frequently lack complete information about protected characteristics (like race, gender, or age) for all individuals in their datasets, making traditional fairness approaches impractical.

Random annotation is inefficient: When organizations manually label only a small portion of data to identify bias, randomly selecting which instances to annotate preserves the original dataset's imbalances, meaning bias mitigation efforts are wasted on redundant information.

Strategic selection of what to annotate matters: The proposed APOD method intelligently chooses which instances to annotate rather than selecting randomly, allowing organizations to achieve comparable bias reduction with significantly fewer labeled examples—potentially reducing costs and privacy risks of collecting sensitive attributes.

Theoretical guarantees on bias reduction are possible: The framework includes mathematical proof that it can bound (limit) algorithmic bias, providing confidence that the approach will actually deliver fairness improvements rather than just claiming to do so.

Practical performance validated across multiple domains: Testing on five different benchmark datasets showed APOD outperformed existing methods under limited annotation budgets, suggesting the approach could be reliably deployed across various real-world applications where complete sensitive attribute data is unavailable.