AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
314 incidents
Excessive Automated Monitoring Alerts Ignored by Staff, Resulting in Private Data Theft of Seventy Million Target Customers
Alerts about a Target data breach were ignored by Minneapolis Target’s staff reportedly due to them being included with many other potential false alerts, and due to some of the company’s network infiltration alerting systems being off to reduce such false alerts, causing private data theft for millions of customers.
Michigan's Unemployment Benefits Algorithm MiDAS Issued False Fraud Claims to Thousands of People
Michigan’s MiDAS system falsely accused over 34,000 people of unemployment fraud from 2013 to 2015, which reportedly caused financial ruin for many. The automated system was designed to cut costs, but it adjudicated fraud cases without human oversight. That led to an 85% error rate. Victims faced wage garnishments, some lost homes, and some faced bankruptcy. Despite early warnings, Michigan’s UIA defended MiDAS until lawsuits and federal pressure forced reforms. Legislators have been seeking compensation for those wrongfully accused.
Compromise of National Biometric ID Card System Leads to Reverification and Change of Status
When the leader of the Afghan Taliban was found possessing a valid ID card in the Pakistani national biometric identification database system, Pakistan launch a national re-verification campaign that is linked to numerous changes in recognition status and loss of services.
NJ Transit's Use of Modeling Software Miscalculated Storm Surge Threat Level
New Jersey Transit's use of a federal government storm modeling software underestimated the threat of storm surges to the Meadows Maintenance Complex, leaving millions of dollars worth of equipment in the rail yard before Hurricane Sandy struck.
Police Departments Reported ShotSpotter as Unreliable and Wasteful
ShotSpotter algorithmic systems locating gunshots were reported by police departments for containing high false positive rates and wasting police resources, prompting discontinuation.
New York City Reportedly Released Disputed Value-Added Teacher Ratings for Thousands of Teachers
After a legal dispute over public-records requests, the New York City Department of Education released Teacher Data Reports that used a value-added statistical model to rate thousands of public-school teachers based on estimated student test-score growth. News outlets published the ratings in February 2012, and teachers, unions, and analysts disputed the scores as imprecise and potentially misleading, citing large error margins, limited samples, and apparent roster-linkage errors.
Image Classification of Battle Tanks
A potentially apocryphal story in which an image classifier was produced to differentiate types of battle tanks, but the resulting model keyed in on environmental attributes rather than tank attributes
Algorithmic Teacher Evaluation Program Reportedly Failed to Improve Student Outcomes and Was Alleged to Have Harmed Teachers
Gates Foundation-funded Intensive Partnerships for Effective Teaching Initiative's algorithmic program to assess teacher performance reportedly failed to achieve its goals for student outcomes, particularly for minority students, and was criticized for potentially causing harm against teachers.
Unverified Misreading by Automated Plate Reader Led to Traffic Stop and Restraint of an Innocent Person at Gunpoint in California
In San Francisco, an automated license plate reader (ALPR) camera misread a number as belonging to a stolen vehicle having the wrong make, but its photo was not visually confirmed by the police due to poor quality and allegedly despite multiple chances prior to making a traffic stop, causing an innocent person to be pulled over at gunpoint and restrained in handcuffed.
Algorithm Assessing Risk Faced by Victims of Gender Violence Misclassified Low-Risk Cases, Allegedly Leading to Homicide of Women and Children in Spain
In Spain, the algorithm that assesses recidivism risk in gender violence, VioGén, has allegedly critically underestimated the level of risk in a series of cases that ended in homicide of women and children since its first deployment.
Patriot Missile System Misclassified US Navy Aircraft, Killing Pilot Upon Approval to Fire
US Navy's Patriot missile system misidentified an American Navy F/A-18C Hornet as an enemy projectile, prompting an operator to fire two missiles at the aircraft, which killed the pilot.
US Air Force's Patriot Missile Mistakenly Launched at Ally Fighter Jet, Killing Two
Acting on the recommendation of their Patriot missile system, American Air Force mistakenly launched the missile at an ally UK Tornado fighter jet, which killed two crew members on board.
Error in Pepsi's Number Generation System Led to Decades-Long Damages in the Philippines
Pepsi's number generation system determining daily winners in its Number Fever promotion in the Philippines mistakenly produced a number held by thousands which resulted in riots, deaths, conspiracy theories, and decades of lawsuits.
Nuclear False Alarm
An alert of five incoming intercontinental ballistic missiles was properly identified as a false-positive by the Soviet Union operator Stanislov Petrov.