AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
961 incidents
ETS Used Allegedly Flawed Voice Recognition Evidence to Accuse and Assess Scale of Cheating, Causing Thousands to be Deported from the UK
International testing organization ETS admits voice recognition as evidence of cheating for thousands of previous TOEIC test-takers that reportedly included wrongfully accused people, causing them to be deported without an appeal process or seeing their incriminating evidence.
Excessive Automated Monitoring Alerts Ignored by Staff, Resulting in Private Data Theft of Seventy Million Target Customers
Alerts about a Target data breach were ignored by Minneapolis Target’s staff reportedly due to them being included with many other potential false alerts, and due to some of the company’s network infiltration alerting systems being off to reduce such false alerts, causing private data theft for millions of customers.
Michigan's Unemployment Benefits Algorithm MiDAS Issued False Fraud Claims to Thousands of People
Michigan’s MiDAS system falsely accused over 34,000 people of unemployment fraud from 2013 to 2015, which reportedly caused financial ruin for many. The automated system was designed to cut costs, but it adjudicated fraud cases without human oversight. That led to an 85% error rate. Victims faced wage garnishments, some lost homes, and some faced bankruptcy. Despite early warnings, Michigan’s UIA defended MiDAS until lawsuits and federal pressure forced reforms. Legislators have been seeking compensation for those wrongfully accused.
Compromise of National Biometric ID Card System Leads to Reverification and Change of Status
When the leader of the Afghan Taliban was found possessing a valid ID card in the Pakistani national biometric identification database system, Pakistan launch a national re-verification campaign that is linked to numerous changes in recognition status and loss of services.
Coffee Meets Bagel’s Algorithm Reported by Users Disproportionately Showing Them Matches of Their Own Ethnicities Despite Selecting “No Preference”
Users selecting “no preference” were shown by Coffee Meets Bagels’s matching algorithm more potential matches with the same ethnicity, which was acknowledged and justified by its founder as a means to maximize connection rate without sufficient user information.
Sexist and Racist Google Adsense Advertisements
Advertisements chosen by Google Adsense are reported as producing sexist and racist results.
UT Austin's GRADE Algorithm Reportedly Reduced Review of Lower-Scored PhD Applicants Amid Bias Concerns
From the 2013 through 2019 admissions cycles, UT Austin's Department of Computer Science used GRADE, a statistical machine-learning system trained on past admissions decisions, to score and organize PhD applications. Critics said the system could reproduce historical admissions inequities and reduce attention to lower-scored applicants, while UT Austin said human reviewers still evaluated each file and later discontinued the tool.
Police Departments Reported ShotSpotter as Unreliable and Wasteful
ShotSpotter algorithmic systems locating gunshots were reported by police departments for containing high false positive rates and wasting police resources, prompting discontinuation.
Chicago Police's Strategic Subject List Reportedly Biased Along Racial Lines
The Chicago Police Department reportedly used person-based predictive policing models, including the Strategic Subject List and Crime and Victimization Risk Model, to assign violence-risk scores to hundreds of thousands of residents. Official and media reports alleged that the program gave police a mathematically fragile basis for treating heavily policed residents as future victims or suspects, with Black Chicago residents especially exposed to the consequences of that scoring.
Police Reportedly Deployed ShotSpotter Sensors Disproportionately in Neighborhoods of Color
Police departments disproportionately placed ShotSpotter sensors in black and brown neighborhoods, which is denounced by communities for allegedly creating dangerous situations, such as one involving in Adam Toledo's death.
New York City Reportedly Released Disputed Value-Added Teacher Ratings for Thousands of Teachers
After a legal dispute over public-records requests, the New York City Department of Education released Teacher Data Reports that used a value-added statistical model to rate thousands of public-school teachers based on estimated student test-score growth. News outlets published the ratings in February 2012, and teachers, unions, and analysts disputed the scores as imprecise and potentially misleading, citing large error margins, limited samples, and apparent roster-linkage errors.
Google Instant's Allegedly 'Anti-Semitic' Results Lead To Lawsuit In France
The organizations SOS Racisme, Union of Jewish Students of France, Movement Against Racism and for Friendship Among Peoples are suing Google due to its autocomplete software suggesting "jewish" when the names of certain public figures were searched on the platform.
Major Universities Are Reportedly Using Race as a 'High Impact Predictor' of Student Success
Several major universities are reportedly using a tool that uses race as one factor to predict student success.
Image Classification of Battle Tanks
A potentially apocryphal story in which an image classifier was produced to differentiate types of battle tanks, but the resulting model keyed in on environmental attributes rather than tank attributes
Apple Tweaked App Store Ranking Algorithms, Allegedly Resulted in Demotion of Local Apps in China
Following Apple’s changes in ranking algorithm in its iTunes App Store, apps by allegedly reputable companies and local startups in China experienced significant drops in ranking order.