Comparing Human and Machine Bias in Face Recognition

Dataset quality matters for bias measurement: The paper identifies that widely-used facial recognition datasets like LFW and CelebA contain significant flaws in their metadata and structure (such as identical images appearing in both test and training sets), which can distort measurements of algorithmic bias and make comparisons unreliable.

AI bias mirrors human bias rather than exceeding it: Computer models showed similar levels of bias as human survey participants on the same tasks, suggesting that facial recognition systems may be learning patterns from human decision-making rather than introducing entirely new forms of discrimination.

Demographic disparities affect both humans and machines: Both algorithms and human reviewers performed significantly worse on dark-skinned and female subjects in facial identification and verification tasks, indicating these are persistent challenges across human and machine recognition rather than unique algorithmic failures.

"Matching" demographics can improve accuracy for both: Both computer models and human participants achieved higher accuracy rates when their own demographics matched those of the people in the images being analyzed, suggesting that familiarity or representation may influence performance across both human and artificial systems.

Machines outperform humans overall, but not without bias: While computer models achieved higher accuracy than human survey participants on both tasks, this superior performance did not eliminate the same demographic-based performance gaps observed in human reviewers, indicating that accuracy improvements alone don't solve bias problems.