AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
961 incidents
YouTube's Algorithms Failed to Remove Violating Content Related to Suicide and Self-Harm
Terms-of-service-violating videos related to suicide and self-harm reportedly bypassed YouTube’s content moderation algorithms, allegedly resulting in exposure of graphic content to young users via recommended videos.
Apple Maps Allegedly Directed Ski Trip Couple Onto Unpaved Road in the Mountains
Near Los Angeles, Apple Maps allegedly directed a couple on a ski trip in the mountains toward into an unconventional route out of town, where the drivers found themselves lost and stuck on an unpaved road in the snow.
YouTube’s Recommendation Algorithm Allegedly Promoted Climate Misinformation Content
YouTube’s recommendation system and its focus on views and watched time were alleged by an advocacy group to have driven people towards climate denial and misinformation videos.
New Jersey Police Wrongful Arrested Innocent Black Man via FRT
Woodbridge Police Department falsely arrested an innocent Black man following a misidentification by their facial recognition software, who was jailed for more than a week and paid thousands of dollar for his defense.
Warehouse robot ruptures can of bear spray and injures workers
Twenty-four Amazon workers in New Jersey were hospitalized after a robot punctured a can of bear repellent spray in a warehouse.
Tumblr Automated Pornography-Detecting Algorithms Erroneously Flagged Inoffensive Images as Explicit
Tumblr’s automated tools to identify adult content were reported to have incorrectly flagged inoffensive images as explicit, following its announcement to ban all adult content on the platform.
Facial Recognition Pilot in Bahia Reportedly Targeted Black and Poor People
Facial recognition deployed in a pilot project by the local government of Bahia despite having minimal hit rate reportedly targeted Black and poor people disproportionately.
Schufa Credit Scoring in Germany Reported for Unreliable and Imbalanced Scores
Creditworthiness Schufa scores in Germany reportedly privileged older and female consumers, and people who changed addresses less frequently, and were unreliable depending on scoring version.
Reported Automated License Plate Reader Alert on Previously Stolen Rental Car Leads to Alleged Wrongful Detainment in California
A purportedly automated license plate reader allegedly flagged a rental car in California as stolen, although reports indicate it had already been recovered. Police reportedly conducted a high-risk stop and detained the occupants at gunpoint before confirming their innocence. The driver later alleged excessive force and database errors in a federal lawsuit.
Emotion Detection Models Showed Disparate Performance along Racial Lines
Emotion detection tools by Face++ and Microsoft's Face API allegedly scored smiling or defaulted ambiguous facial photos for Black faces as negative emotion more often than for white faces.
Wrongful Attempted Arrest for Apple Store Thefts Due to NYPD’s Facial Misidentification
New York Police Department (NYPD)’s facial recognition system falsely connected a Black teenager to a series of thefts at Apple stores, which resulted in his wrongful attempted arrest.
Picture of Woman on Side of Bus Shamed for Jaywalking
Facial recognition system in China mistakes celebrity's face on moving billboard for jaywalker
Crashes with Maneuvering Characteristics Augmentation System (MCAS)
A Boeing 737 crashed into the sea, killing 189 people, after faulty sensor data caused an automated manuevering system to repeatedly push the plane's nose downward.
Facebook "News Feed" Allegedly Boosted Misinformation and Violating Content Following Use of MSI Metric
After the “News Feed” algorithm had been overhauled to boost engagement between friends and family in early 2018, its heavy weighting of re-shared content was alleged found by company researchers to have pushed content creators to reorient their posts towards outrage and sensationalism, causing a proliferation of misinformation, toxicity, and violent content.
Oregon’s Screening Tool for Child Abuse Cases Discontinued Following Concerns of Racial Bias
Oregon’s Department of Human Services (DHS) stopped using its Safety at Screening Tool, that is aimed to predict the risk that children wind up in foster care or be investigated in the future, and opted for a new process allegedly to reduce disparities and improve racially equitable decision-making.