AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
682 incidents
UK passport photo checker shows bias against dark-skinned women
UK passport photo checker shows bias against dark-skinned women.
Facebook’s Algorithm Mistook an Advertisement of Onions as Sexual Suggestive Content
Facebook’s content moderation algorithm misidentified and removed a Canadian business’s advertisement containing a photo of onions as products of overtly sexual content, which was later reinstated after review.
Amazon’s Robotic Fulfillment Centers Have Higher Serious Injury Rates
Amazon’s robotic fulfillment centers have higher serious injury rates.
Twitter’s Image Cropping Tool Allegedly Showed Gender and Racial Bias
Twitter's photo cropping algorithm was revealed by researchers to favor white and women faces in photos containing multiple faces, prompting the company to stop its use on mobile platform.
Philosophy AI Tentatively Produced Offensive Results for Certain Prompts
Philosopher AI as built on top of GPT-3 was reported by its users for having strong tendencies to produce offensive results when given prompts on certain topics such as feminism and Ethiopia.
Philosophy AI Allegedly Used To Generate Mixture of Innocent and Harmful Reddit Posts
Philosopher AI, a GPT-3-powered controversial text generator, was allegedly used by an anonymous actor on AskReddit subreddit, whose posts featured a mixture of harmless stories, conspiracy theories, and sensitive topic discussions.
OpenAI's GPT-3 Associated Muslims with Violence
Users and researchers revealed generative AI GPT-3 associating Muslims to violence in prompts, resulting in disturbingly racist and explicit outputs such as casting Muslim actor as a terrorist.
Bug in Instagram’s “Related Hashtags” Algorithm Allegedly Caused Disproportionate Treatment of Political Hashtags
A bug was reported by Instagram’s spokesperson to have prevented an algorithm from populating related hashtags for thousands of hashtags, resulting in an allege preferential treatment for some politically partisan hashtags.
Colorado Police’s Automated License Plate Reader (ALPR) Matched a Family’s Minivan’s Plate to That of a Stolen Vehicle Allegedly, Resulting in Detainment at Gunpoint
An automated plate reader reportedly matched a license plate information, but of a family’s minivan and an alleged motorcycle in Montana that was reportedly stolen earlier in the year, resulting in them and their children being held at gunpoint and detained in handcuffs by multiple Aurora police officers.
Genderify’s AI to Predict a Person’s Gender Revealed by Free API Users to Exhibit Bias
A company's AI predicting a person's gender based on their name, email address, or username was reported by its users to show biased and inaccurate results.
Facebook Provided Offensive Translation for King of Thailand's Birthday Ceremony
Facebook's Thai-English translation gave an inappropriate mistranslation on Thai PBS's Facebook live broadcast of the King of Thailand’s candle-lighting birthday ceremony.
Thoughts App Allegedly Created Toxic Tweets
Tweets created by Thoughts, a tweet generation app that leverages OpenAI’s GPT-3, allegedly exhibited toxicity when given prompts related to minority groups.
International Baccalaureate’s 2020 Calculated-Grades Model Reportedly Lowered Students' Scores and College Prospects
After Covid-19 canceled May 2020 exams, the Geneva-based International Baccalaureate reportedly used a calculated-grades model based on coursework, predicted grades, and historical school results to estimate final marks. Students, parents, and teachers reportedly said the opaque model produced unexpectedly low scores, threatening university admissions, scholarships, and study plans across multiple countries.
NarxCare’s Risk Score Model Allegedly Lacked Validation and Trained on Data with High Risk of Bias
NarxCare's overdose risk algorithm, lacking peer-reviewed validation, uses sensitive data like doctor visits, prescriptions, and possibly genetic information, leading to significant biases against women and Black patients. Factors like sexual abuse and criminal records exacerbate stigmas and disparities, often resulting in unjust denial of necessary pain medication. The newly approved AvertD genetic test shares similar issues, further complicating and potentially harming medical treatment decisions.
Facebook’s Political Ad Detection Reportedly Showed High and Geographically Uneven Error Rates
The performance of Facebook’s political ad detection was revealed by researchers to be imprecise, uneven across countries in errors, and inadequate for preventing systematic violations of political advertising policies.