AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
1,751 incidents
BBC Reporter's Twin Brother Cracked HSBC's Voice ID Authentication
HSBC’s voice recognition authentication system was fooled after seven repeated attempts by a BBC reporter's twin brother who mimicked his voice to access his bank account.
Houston ISD's EVAAS Teacher-Evaluation System Reportedly Put Teachers' Jobs at Risk Through Unverifiable Scores
From 2011 to 2015, Houston Independent School District used SAS Institute’s Education Value-Added Assessment System (EVAAS) to rate teacher effectiveness from student test-score growth. Teachers alleged that proprietary, unverifiable EVAAS scores contributed to terminations and contract nonrenewals. On May 4, 2017, a federal judge allowed due-process claims over the system to proceed.
Overfit Kaggle Models Discouraged Data Science Competitors
In the “The Nature Conservancy Fisheries Monitoring” competition on the data science competition website Kaggle, a number of competitors overfit their image classifier models to a poorly representative validation data set.
New York Detective Reportedly Misused Woody Harrelson's Face to Perform Face Recognition Search
When the facial recognition search for a CVS theft suspect's face returned no useful matches due to the surveillance footage being obscured and highly pixelated, a New York City police detective continued the face search using Woody Harrelson's face allegedly due to his resemblance to the suspect's face, eventually leading to the arrest of an unknown victim.
FaceApp Racial Filters
FaceApp is criticized for offering racist filters.
Facebook Reportedly Outed Sex Workers through Friend Recommendations
Facebook's "People You May Know" feature reportedly outed sex workers by recommending clients to their personal accounts or family members to their business accounts with no option to opt out.
Gender Biases in Google Translate
A Cornell University study in 2016 highlighted Google Translate's pattern of assigning gender to occupations in a way showing an implicit gender bias against women.
Pennsylvania County's Family Screening Tool Allegedly Exhibited Discriminatory Effects
Data analysis by the American Civil Liberty Union (ACLU) on Allegheny County's decision-support Family Screening Tool to predict child abuse or neglect risk found the tool resulting in higher screen-in rates for Black families and higher risk scores for households with disabled residents.
IBM Watson for Oncology Criticized by Customers for Allegedly Unsafe and Inaccurate Cancer Treatment Recommendations
Internal documents from IBM Watson Health showed negative assessments from customers such as Florida’s Jupiter Hospital and Memorial Sloan Kettering criticizing its Watson for Oncology product for allegedly unsafe and incorrect cancer treatment recommendations.
Nissan's "Automatic Emergency Braking" False Positives Posed Traffic Risks to Drivers
Nissan's Automatic Emergency Braking (AEB) feature was reported in a series of complaints for false positives and abrupt braking behaviors, endangering car occupants and traffic participants.
Social Media's Automated Word-Flagging without Context Shifted Content Creators' Language Use
TikTok's, YouTube's, Instagram's, and Twitch's use of algorithms to flag certain words devoid of context changed content creators' use of everyday language or discussion about certain topics in fear of their content getting flagged or auto-demonetized by mistake.
Uber Locked Indian Drivers out of Accounts Allegedly Due to Facial Recognition Fails
Uber drivers in India reported being locked out of their accounts allegedly due to Real-Time ID Check's facial recognition failing to recognize appearance changes or faces in low lighting conditions.
Male College Freshman Allegedly Made Porn Deepfakes Using Female Friend's Face
A female college student's face was superimposed on another woman's body in deepfake pornographic videos and shared on 4chan allegedly by a male student whose friendship with her fell apart during freshman year.
High-Toxicity Assessed on Text Involving Women and Minority Groups
Google's Perspective API, which assigns a toxicity score to online text, seems to award higher toxicity scores to content involving non-white, male, Christian, heterosexual phrases.
Wikipedia Vandalism Prevention Bot Loop
Wikipedia bots meant to remove vandalism clash with each other and form feedback loops of repetitve undoing of the other bot's edits.