BiasFairnessAccountabilitycs.CY
Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval
Kyra Wilson, Aylin Caliskan
July 29, 2024
AAAI/ACM Conference on AI, Ethics, and Society
145 citations
9 influential
Key Takeaways
-
AI hiring tools show significant racial bias: The study found that language models used for resume screening favor White-associated names in 85.1% of cases across nine different occupations, meaning qualified candidates from racial minorities are systematically disadvantaged by these automated systems.
-
Gender bias patterns are complex and occupation-dependent: Contrary to some expectations, female-associated names were favored in only 11.1% of cases overall, suggesting that gender bias in AI hiring tools varies significantly depending on the job type rather than showing a consistent pattern.
-
Black men face compounded disadvantage: The research validated "intersectionality" concerns by showing that Black male candidates faced discrimination in up to 100% of cases in some occupations—demonstrating that biases based on race and gender can combine to create even greater barriers for certain groups.
-
AI bias mirrors real-world employment discrimination: The patterns of bias found in these language models replicate documented discrimination in actual hiring practices, suggesting that AI systems are learning and amplifying existing societal inequities rather than providing neutral, objective screening.
-
Technical factors influence bias outcomes: The study identified that seemingly neutral factors like resume length and how common a name is in training data affect which candidates get selected, highlighting that bias in AI hiring isn't always intentional but emerges from how these systems are built and trained.