BiasFairnessAccountabilitycs.CY
No Thoughts Just AI: Biased LLM Hiring Recommendations Alter Human Decision Making and Limit Human Autonomy
Kyra Wilson, Mattea Sim, Anna-Maria Gueorguieva, Aylin Caliskan
September 4, 2025
AAAI/ACM Conference on AI, Ethics, and Society
23 citations
1 influential
Key Takeaways
-
AI bias dramatically influences human hiring decisions: When AI systems showed race-based preferences, participants selected favored candidates up to 90% of the time, compared to equal selection rates when AI showed no bias—demonstrating that biased AI recommendations can override human judgment even when people are aware they're evaluating resumes.
-
Human autonomy is compromised regardless of perceived AI quality: Participants' hiring decisions remained vulnerable to AI bias "even if people think AI recommendations are low quality or not important," suggesting that the influence of biased AI is difficult to resist through conscious effort alone.
-
Awareness of personal bias can partially counteract AI influence: Completing an Implicit Association Test (IAT) before resume screening increased selection of candidates from underrepresented groups by 13%, indicating that prompting people to reflect on their own unconscious associations may provide some protection against adopting AI's biases.
-
Biased AI perpetuates racial stereotypes in hiring: The study found that AI bias particularly affected selection of candidates whose identities don't align with common race-status stereotypes, meaning biased hiring AI actively reinforces existing workplace inequalities rather than simply reflecting them.
-
Current AI hiring systems pose significant risks to fair employment: The research demonstrates that AI-assisted hiring—even with moderate bias levels—can systematically disadvantage candidates from certain racial groups at scale across multiple occupations and scenarios.