AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
314 incidents
Security Robot Drowns Itself in a Fountain
A Knightscope K5 security robot ran itself into a water fountain in Washington, DC.
Atlas Robot Fell off Stage at Conference
Boston Dynamics’s autonomous robot Atlas allegedly caught its foot on a stage light, resulting in a fall off the stage at the Congress of Future Science and Technology Leaders conference.
High False Positive Rate by SWP's Facial Recognition Use at Champion's League Final
South Wales Police (SWP)’s automated facial recognition (AFR) at the Champion's League Final football game in Cardiff wrongly identified innocent people as potential matches at an extremely high false positive rate of more than 90%.
Houston ISD's EVAAS Teacher-Evaluation System Reportedly Put Teachers' Jobs at Risk Through Unverifiable Scores
From 2011 to 2015, Houston Independent School District used SAS Institute’s Education Value-Added Assessment System (EVAAS) to rate teacher effectiveness from student test-score growth. Teachers alleged that proprietary, unverifiable EVAAS scores contributed to terminations and contract nonrenewals. On May 4, 2017, a federal judge allowed due-process claims over the system to proceed.
Overfit Kaggle Models Discouraged Data Science Competitors
In the “The Nature Conservancy Fisheries Monitoring” competition on the data science competition website Kaggle, a number of competitors overfit their image classifier models to a poorly representative validation data set.
New York Detective Reportedly Misused Woody Harrelson's Face to Perform Face Recognition Search
When the facial recognition search for a CVS theft suspect's face returned no useful matches due to the surveillance footage being obscured and highly pixelated, a New York City police detective continued the face search using Woody Harrelson's face allegedly due to his resemblance to the suspect's face, eventually leading to the arrest of an unknown victim.
IBM Watson for Oncology Criticized by Customers for Allegedly Unsafe and Inaccurate Cancer Treatment Recommendations
Internal documents from IBM Watson Health showed negative assessments from customers such as Florida’s Jupiter Hospital and Memorial Sloan Kettering criticizing its Watson for Oncology product for allegedly unsafe and incorrect cancer treatment recommendations.
Nissan's "Automatic Emergency Braking" False Positives Posed Traffic Risks to Drivers
Nissan's Automatic Emergency Braking (AEB) feature was reported in a series of complaints for false positives and abrupt braking behaviors, endangering car occupants and traffic participants.
Wikipedia Vandalism Prevention Bot Loop
Wikipedia bots meant to remove vandalism clash with each other and form feedback loops of repetitve undoing of the other bot's edits.
Honda's CMBS False Positives Allegedly Caused Accidents to Customers
Honda's Collision Mitigation Braking System (CMBS) allegedly caused accidents to consumers due to frequent instances of false obstacle detection.
Reinforcement Learning Reward Functions in Video Games
OpenAI published a post about its findings when using Universe, a software for measuring and training AI agents to conduct reinforcement learning experiments, showing that the AI agent did not act in the way intended to complete a videogame.
Robot at a Chinese Tech Fair Smashed a Glass Booth, Injuring a Visitor
At the 18th China Hi-Tech Fair, a robot suddenly smashed through a glass booth and injured a visitor, after a staff member reportedly mistakenly pressed a button, causing it to reverse and accelerate.
Poor Performance of Tesla Factory Robots
The goal of manufacturing 2,500 Tesla Model 3's per week was falling short by 500 cars/week, and employees had to be "borrowed" from Panasonic in a shared factory to help hand-assemble lithium batteries for Tesla.
Tesla FSD Reportedly Increased Crash Risk, Prompting Recall
A component of Tesla Full Self Driving system was deemed by regulators to increase crash risk such as by exceeding speed limits or by traveling through intersections unlawfully or unpredictably, prompting recall for hundreds of thousands of vehicles.
Tougher Turing Test Exposes Chatbots’ Stupidity (migrated to Issue)
The 2016 Winograd Schema Challenge highlighted how even the most successful AI systems entered into the Challenge were only successful 3% more often than random chance. This incident has been downgraded to an issue as it does not meet current ingestion criteria.