AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
244 incidents
Detroit Police Wrongfully Arrested Black Man Due To Faulty Facial Recognition
A Black man was wrongfully detained by the Detroit Police Department as a result of a false facial recognition (FRT) result.
Amazon’s "Time Off Task" System Made False Assumptions about Workers' Time Management
Amazon’s warehouse worker “time off task" (TOT) tracking system was used to discipline and dismiss workers, falsely assuming workers to have wasted time and failing to account for breaks or equipment issues.
TikTok’s “For You” Algorithm Exposed Young Users to Pro-Eating Disorder Content
TikTok’s young users were allegedly exposed to community-guideline-violating pro-eating disorder content on their algorithmically curated “For You” page that serves videos from any user on its platform.
Facebook Allegedly Failed to Police Hate Speech Content That Contributed to Ethnic Violence in Ethiopia
Facebook allegedly did not adequately remove hate speech, some of which was extremely violent and dehumanizing, on its platform including through automated means, contributing to the violence faced by ethnic communities in Ethiopia.
Workday's AI Tools Allegedly Enabled Employers to Discriminate against Applicants of Protected Groups
Workday's algorithmic screening systems were alleged in a lawsuit allowing employers to discriminate against African-Americans, people over 40, and people with disabilities.
Algorithm to Distribute Social Welfare Reported for Oversimplifying Economic Vulnerability
Takaful cash transfer program's algorithm which ranks families by their economic vulnerability level to determine financial assistance reportedly oversimplified people's economic situation, fueling social tension and perceptions of unfairness.
Facebook's Ad Delivery Reportedly Excluded Audience along Racial and Gender Lines
Facebook's housing and employment ad delivery process allegedly resulted in skews in exposure for some users along demographic lines such as gender and racial identity.
The Christchurch shooter and YouTube’s radicalization trap
A New Zealand government report released following a right-wing terrorist killing 51 worshippers at two New Zealand mosques which indicated that Youtube's recommendation algorithm played an important role in the terrorist's radicalization.
YouTube's Algorithms Failed to Remove Violating Content Related to Suicide and Self-Harm
Terms-of-service-violating videos related to suicide and self-harm reportedly bypassed YouTube’s content moderation algorithms, allegedly resulting in exposure of graphic content to young users via recommended videos.
New Jersey Police Wrongful Arrested Innocent Black Man via FRT
Woodbridge Police Department falsely arrested an innocent Black man following a misidentification by their facial recognition software, who was jailed for more than a week and paid thousands of dollar for his defense.
Facial Recognition Pilot in Bahia Reportedly Targeted Black and Poor People
Facial recognition deployed in a pilot project by the local government of Bahia despite having minimal hit rate reportedly targeted Black and poor people disproportionately.
Schufa Credit Scoring in Germany Reported for Unreliable and Imbalanced Scores
Creditworthiness Schufa scores in Germany reportedly privileged older and female consumers, and people who changed addresses less frequently, and were unreliable depending on scoring version.
Emotion Detection Models Showed Disparate Performance along Racial Lines
Emotion detection tools by Face++ and Microsoft's Face API allegedly scored smiling or defaulted ambiguous facial photos for Black faces as negative emotion more often than for white faces.
Facebook "News Feed" Allegedly Boosted Misinformation and Violating Content Following Use of MSI Metric
After the “News Feed” algorithm had been overhauled to boost engagement between friends and family in early 2018, its heavy weighting of re-shared content was alleged found by company researchers to have pushed content creators to reorient their posts towards outrage and sensationalism, causing a proliferation of misinformation, toxicity, and violent content.
Oregon’s Screening Tool for Child Abuse Cases Discontinued Following Concerns of Racial Bias
Oregon’s Department of Human Services (DHS) stopped using its Safety at Screening Tool, that is aimed to predict the risk that children wind up in foster care or be investigated in the future, and opted for a new process allegedly to reduce disparities and improve racially equitable decision-making.