AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
1,751 incidents
Sleeping Driver on Tesla AutoPilot
A Tesla Model S remained on autopilot while being operated by a drunk, sleeping operator whose hands were not on the wheel. The police had to slow the car down by slowing in front of the vehicle to activate its 'driver assist' feature .
Schufa Credit Scoring in Germany Reported for Unreliable and Imbalanced Scores
Creditworthiness Schufa scores in Germany reportedly privileged older and female consumers, and people who changed addresses less frequently, and were unreliable depending on scoring version.
Reported Automated License Plate Reader Alert on Previously Stolen Rental Car Leads to Alleged Wrongful Detainment in California
A purportedly automated license plate reader allegedly flagged a rental car in California as stolen, although reports indicate it had already been recovered. Police reportedly conducted a high-risk stop and detained the occupants at gunpoint before confirming their innocence. The driver later alleged excessive force and database errors in a federal lawsuit.
Emotion Detection Models Showed Disparate Performance along Racial Lines
Emotion detection tools by Face++ and Microsoft's Face API allegedly scored smiling or defaulted ambiguous facial photos for Black faces as negative emotion more often than for white faces.
Wrongful Attempted Arrest for Apple Store Thefts Due to NYPD’s Facial Misidentification
New York Police Department (NYPD)’s facial recognition system falsely connected a Black teenager to a series of thefts at Apple stores, which resulted in his wrongful attempted arrest.
Picture of Woman on Side of Bus Shamed for Jaywalking
Facial recognition system in China mistakes celebrity's face on moving billboard for jaywalker
Crashes with Maneuvering Characteristics Augmentation System (MCAS)
A Boeing 737 crashed into the sea, killing 189 people, after faulty sensor data caused an automated manuevering system to repeatedly push the plane's nose downward.
Facebook "News Feed" Allegedly Boosted Misinformation and Violating Content Following Use of MSI Metric
After the “News Feed” algorithm had been overhauled to boost engagement between friends and family in early 2018, its heavy weighting of re-shared content was alleged found by company researchers to have pushed content creators to reorient their posts towards outrage and sensationalism, causing a proliferation of misinformation, toxicity, and violent content.
Oregon’s Screening Tool for Child Abuse Cases Discontinued Following Concerns of Racial Bias
Oregon’s Department of Human Services (DHS) stopped using its Safety at Screening Tool, that is aimed to predict the risk that children wind up in foster care or be investigated in the future, and opted for a new process allegedly to reduce disparities and improve racially equitable decision-making.
Dutch Families Wrongfully Accused of Tax Fraud Due to Discriminatory Algorithm
A childcare benefits system in the Netherlands falsely accused thousands of families of fraud, in part due to an algorithm that treated having a second nationality as a risk factor.
Facebook Allegedly Failed to Police Anti-Rohingya Hate Speech Content That Contributed to Violence in Myanmar
Facebook allegedly did not adequately remove anti-Rohingya hate speech, some of which was extremely violent and dehumanizing, on its platform, contributing to the violence faced by Rohingya communities in Myanmar.
HUD charges Facebook with enabling housing discrimination
In March 2019 the U.S. Department of Housing and Urban Development charged Facebook with violating the Fair Housing Act by allowing real estate sellers to target advertisements in a discriminatory manner.
Amazon Allegedly Tweaked Search Algorithm to Boost Its Own Products
Amazon tweaked product-search algorithm to boost and guide customers towards more profitable in-house products instead of showing mainly most-relevant and best-selling listings, which its internal engineers and lawyers alleged to violate company’s best-for-customer principle.
Amazon's Rekognition Falsely Matched Members of Congress to Mugshots
Rekognition's face comparison feature was shown by the ACLU to have misidentified members of congress, and particularly members of colors, as other people who have been arrested using a mugshot database built on publicly available arrest photos.
Swedish Contraceptive App, Natural Cycles, Allegedly Failed to Correctly Map Menstrual Cycle
Some women using the contraceptive app, Natural Cycles, reported unwanted pregnancies, revealing its algorithm's difficulties in mapping menstrual cycles.