AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
14 incidents
Texas Homeowner Reportedly Spent $3,000 to Contest AI-Flagged Warning of Insurance Nonrenewal
In Texas, homeowners including Tracy Gartenmann reported warnings against insurance nonrenewals based on aerial imagery flagged by AI systems. Gartenmann's insurer, Travelers, reportedly cited overhanging trees. She reportedly spent $3,000 to address the issue and retain coverage. While the AI system appears to have functioned as intended, critics argue it lacks adequate human oversight and imposes material and emotional burdens on policyholders. Other cases reportedly involved disputed roof assessments. Companies such as State Farm and Nationwide used vendors like CAPE Analytics and Nearmap.
Names Linked to Defamation Lawsuits Reportedly Spur Filtering Errors in ChatGPT's Name Recognition
ChatGPT has reportedly been experiencing errors and service disruptions caused by hard-coded filters designed to prevent it from producing potentially harmful or defamatory content about certain individuals by blocking prompts containing specific names, likely related to post-training interventions. The reported names are Brian Hood, Jonathan Turley, Jonathan Zittrain, David Faber, David Mayer, and Guido Scorza.
California Homeowner Reportedly Loses Insurance After Purported Aerial Imagery-Based Roof Assessment
A California homeowner's insurance policy was reportedly not renewed after CSAA relied on aerial imagery, purportedly analyzed by an AI system, to assess the roof as having reached the end of its life. The homeowner, Cindy Picos, reportedly commissioned an independent inspection suggesting the roof had a decade of remaining life, but CSAA declined to reverse the decision. The insurer reportedly did not provide the imagery for review.
Cigna Algorithm PXDX Allegedly Rejected Thousands of Patient Claims En Masse in Breach of California Law
Cigna health insurer faces a class-action lawsuit for allegedly using the PXDX ("procedure-to-diagnosis") algorithm to automatically reject over 300,000 patient claims in violation of California law, prompting two members to file the lawsuit seeking damages and a jury trial. Cigna disputes the allegations, claiming the process expedites physician reimbursement and does not result in care denials.
UnitedHealth Accused of Deploying Allegedly Flawed AI to Deny Medical Coverage
UnitedHealthcare allegedly used a faulty AI algorithm with a 90% error rate to override doctors' recommendations and deny health coverage. This AI, developed by NaviHealth, reportedly led to premature discharge from care facilities and substantial out-of-pocket expenses for patients, according to a lawsuit filed in the District Court for Minnesota.
Google’s Assistive Writing Feature Provided Allegedly Unnecessary and Clumsy Suggestions
Google’s “inclusive language” feature prompting writers to consider alternatives to non-inclusive words reportedly also recommend alternatives for words such as “landlord” and “motherboard,” which critics said was a form of obtrusive, unnecessary, and bias-reinforcing speech-policing.
Uber Launched Opaque Algorithm That Changes Drivers' Payments in the US
Uber launched a new but opaque algorithm to determine drivers' pay in the US which allegedly caused drivers to experience lower fares, confusing fare drops, and a decrease in rides.
Algorithmic Allocation of Resources in Healthcare for Disabled and Elderly Care Services Allegedly Harmed Patients
A healthcare algorithm designed to equitably distribute caregiving resources is alleged to have drastically cut care hours for the disabled and elderly, purportedly leading to significant hardships and harm. Initially developed for fair resource allocation, the system reportedly faced legal challenges for its inability to accurately assess individual needs, resulting in allegedly reduced essential care.
Algorithmic Staffing Failures Linked to Resident Deaths at Leading Assisted-Living Chain Brookdale
Brookdale Senior Living's algorithm-based staffing system, "Service Alignment," reportedly left facilities understaffed, leading to critical incidents. For example, on April 21, 2021, Louise Walker, a resident at Brookdale's Jacksonville facility, died after falling and being left unattended for over two hours. State investigators cited Brookdale for medical neglect. The algorithm has been linked to multiple incidents of neglect, injuries, and deaths, prompting lawsuits.
Tesla Disabled “Rolling Stop” Functionality Associated with the “Aggressive” Driving Mode
The “rolling stop” functionality within the “Aggressive” Full Self Driving (FSD) profile that was released via a Tesla firmware update was recalled and disabled.
Length of Stay False Diagnosis Cut Off Insurer's Payment for Treatment of Elderly Woman
An elderly Wisconsin woman was algorithmically determined to have a rapid recovery, an output which the insurer based on to cut off payment for her treatment despite medical notes showing her still experiencing debilitating pain.
Houston ISD's EVAAS Teacher-Evaluation System Reportedly Put Teachers' Jobs at Risk Through Unverifiable Scores
From 2011 to 2015, Houston Independent School District used SAS Institute’s Education Value-Added Assessment System (EVAAS) to rate teacher effectiveness from student test-score growth. Teachers alleged that proprietary, unverifiable EVAAS scores contributed to terminations and contract nonrenewals. On May 4, 2017, a federal judge allowed due-process claims over the system to proceed.
Predictive Policing Program by Florida Sheriff’s Office Allegedly Violated Residents’ Rights and Targeted Children of Vulnerable Groups
The Intelligence-Led Policing model rolled out by the Pasco County Sheriff’s Office was allegedly developed based on flawed science and biased data that also contained sensitive information and irrelevant attributes about students, which critics said to be discriminatory.
New York City Reportedly Released Disputed Value-Added Teacher Ratings for Thousands of Teachers
After a legal dispute over public-records requests, the New York City Department of Education released Teacher Data Reports that used a value-added statistical model to rate thousands of public-school teachers based on estimated student test-score growth. News outlets published the ratings in February 2012, and teachers, unions, and analysts disputed the scores as imprecise and potentially misleading, citing large error margins, limited samples, and apparent roster-linkage errors.