AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
741 incidents
NJ Transit's Use of Modeling Software Miscalculated Storm Surge Threat Level
New Jersey Transit's use of a federal government storm modeling software underestimated the threat of storm surges to the Meadows Maintenance Complex, leaving millions of dollars worth of equipment in the rail yard before Hurricane Sandy struck.
UT Austin's GRADE Algorithm Reportedly Reduced Review of Lower-Scored PhD Applicants Amid Bias Concerns
From the 2013 through 2019 admissions cycles, UT Austin's Department of Computer Science used GRADE, a statistical machine-learning system trained on past admissions decisions, to score and organize PhD applications. Critics said the system could reproduce historical admissions inequities and reduce attention to lower-scored applicants, while UT Austin said human reviewers still evaluated each file and later discontinued the tool.
Police Departments Reported ShotSpotter as Unreliable and Wasteful
ShotSpotter algorithmic systems locating gunshots were reported by police departments for containing high false positive rates and wasting police resources, prompting discontinuation.
Chicago Police's Strategic Subject List Reportedly Biased Along Racial Lines
The Chicago Police Department reportedly used person-based predictive policing models, including the Strategic Subject List and Crime and Victimization Risk Model, to assign violence-risk scores to hundreds of thousands of residents. Official and media reports alleged that the program gave police a mathematically fragile basis for treating heavily policed residents as future victims or suspects, with Black Chicago residents especially exposed to the consequences of that scoring.
Police Reportedly Deployed ShotSpotter Sensors Disproportionately in Neighborhoods of Color
Police departments disproportionately placed ShotSpotter sensors in black and brown neighborhoods, which is denounced by communities for allegedly creating dangerous situations, such as one involving in Adam Toledo's death.
Google Instant's Allegedly 'Anti-Semitic' Results Lead To Lawsuit In France
The organizations SOS Racisme, Union of Jewish Students of France, Movement Against Racism and for Friendship Among Peoples are suing Google due to its autocomplete software suggesting "jewish" when the names of certain public figures were searched on the platform.
Image Classification of Battle Tanks
A potentially apocryphal story in which an image classifier was produced to differentiate types of battle tanks, but the resulting model keyed in on environmental attributes rather than tank attributes
Defamation via AutoComplete
Google's autocomplete feature alongside its image search results resulted in the defamation of people and businesses.
Algorithmic Teacher Evaluation Program Reportedly Failed to Improve Student Outcomes and Was Alleged to Have Harmed Teachers
Gates Foundation-funded Intensive Partnerships for Effective Teaching Initiative's algorithmic program to assess teacher performance reportedly failed to achieve its goals for student outcomes, particularly for minority students, and was criticized for potentially causing harm against teachers.
Unverified Misreading by Automated Plate Reader Led to Traffic Stop and Restraint of an Innocent Person at Gunpoint in California
In San Francisco, an automated license plate reader (ALPR) camera misread a number as belonging to a stolen vehicle having the wrong make, but its photo was not visually confirmed by the police due to poor quality and allegedly despite multiple chances prior to making a traffic stop, causing an innocent person to be pulled over at gunpoint and restrained in handcuffed.
IRS Audited Black Taxpayers More Frequently Reportedly Due to Algorithm
The IRS was auditing Black taxpayers more frequently than other groups allegedly due to the design of their algorithms, focusing on easier-to-conduct audits which inadvertently correlated with the group's pattern of tax filing errors.
Machine Personal Assistants Failed to Maintain Social Norms
During an experiment of software personal assistants at the Information Sciences Institute (ISI) at the University of Southern California (USC), researchers found that the assistants violated the privacy of their principals and were unable to respect the social norms of the office.
Algorithm Assessing Risk Faced by Victims of Gender Violence Misclassified Low-Risk Cases, Allegedly Leading to Homicide of Women and Children in Spain
In Spain, the algorithm that assesses recidivism risk in gender violence, VioGén, has allegedly critically underestimated the level of risk in a series of cases that ended in homicide of women and children since its first deployment.
Automated Adult Content Detection Tools Showed Bias against Women Bodies
Automated content moderation tools to detect sexual explicitness or "raciness" reportedly exhibited bias against women bodies, resulting in suppression of reach despite not breaking platform policies.
Virginia Courts’ Algorithmic Recidivism Risk Assessment Failed to Lower Incarceration Rates
Virginia courts’ use of algorithmic predictions of future offending risks were found by researchers failing to reduce incarceration rates, showed racial and age disparities in risk scores and its application, and neither exacerbated or ameliorated historical racial differences in sentencing.