AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
244 incidents
UK passport photo checker shows bias against dark-skinned women
UK passport photo checker shows bias against dark-skinned women.
Suicide Clips Evaded TikTok's Automated Moderation in Coordinated Attack
Many clips showing a suicide evaded TikTok's automated content moderation system allegedly in a coordinated attack, which resulted in exposure of violating content to its users.
Twitter’s Image Cropping Tool Allegedly Showed Gender and Racial Bias
Twitter's photo cropping algorithm was revealed by researchers to favor white and women faces in photos containing multiple faces, prompting the company to stop its use on mobile platform.
Philosophy AI Tentatively Produced Offensive Results for Certain Prompts
Philosopher AI as built on top of GPT-3 was reported by its users for having strong tendencies to produce offensive results when given prompts on certain topics such as feminism and Ethiopia.
UK Ofqual's Algorithm Disproportionately Provided Lower Grades Than Teachers' Assessments
UK Office of Qualifications and Examinations Regulation (Ofqual)'s grade-standardization algorithm providing predicted grades for A level and GCSE qualifications in the UK, Wales, Northern Ireland, and Scotland was reportedly giving grades lower than teachers' assessments, and disproportionately for state schools.
Activists Allege NYPD's Application of Facial Recognition Interfered with Right to Protest
Black Lives Matter activists alleged being targeted for arrest by New York Police using facial recognition, interfering with their right to protest.
OpenAI's GPT-3 Associated Muslims with Violence
Users and researchers revealed generative AI GPT-3 associating Muslims to violence in prompts, resulting in disturbingly racist and explicit outputs such as casting Muslim actor as a terrorist.
Bug in Instagram’s “Related Hashtags” Algorithm Allegedly Caused Disproportionate Treatment of Political Hashtags
A bug was reported by Instagram’s spokesperson to have prevented an algorithm from populating related hashtags for thousands of hashtags, resulting in an allege preferential treatment for some politically partisan hashtags.
Genderify’s AI to Predict a Person’s Gender Revealed by Free API Users to Exhibit Bias
A company's AI predicting a person's gender based on their name, email address, or username was reported by its users to show biased and inaccurate results.
Thoughts App Allegedly Created Toxic Tweets
Tweets created by Thoughts, a tweet generation app that leverages OpenAI’s GPT-3, allegedly exhibited toxicity when given prompts related to minority groups.
Facebook's AI Put "Primates" Label on Video Featuring Black Men
Facebook's AI mislabeled video featuring Black men as a video about "primates," resulting in an offensive prompt message for users who watched the video.
Image Upscaling Algorithm PULSE Allegedly Produced Facial Images with Caucasian Features More Often
Image upscaling tool PULSE powered by NVIDIA's StyleGAN reportedly generated faces with Caucasian features more often, although AI academics, engineers, and researchers were not in agreement about where the source of bias was.
iGPT, SimCLR Learned Biased Associations from Internet Training Data
Unsupervised image generation models trained using Internet images such as iGPT and SimCLR were shown to have embedded racial, gender, and intersectional biases, resulting in stereotypical depictions.
Facebook’s Moderation Algorithm Banned Users for Historical Evidence of Slavery
Facebook’s automated content moderation was acknowledged by a company spokesperson to have erroneously censored and banned Australian users from posting an article containing a 1890s photo of Aboriginal men in chains over nudity as historical evidence of slavery in Australia.
Microsoft’s Algorithm Allegedly Selected Photo of the Wrong Mixed-Race Person Featured in a News Story
A news story published on MSN.com featured a photo of the wrong mixed-race person that was allegedly selected by an algorithm, following Microsoft’s layoff and replacement of journalists and editorial workers at its organizations with AI systems.