AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
18 incidents
Texas Homeowner Reportedly Spent $3,000 to Contest AI-Flagged Warning of Insurance Nonrenewal
In Texas, homeowners including Tracy Gartenmann reported warnings against insurance nonrenewals based on aerial imagery flagged by AI systems. Gartenmann's insurer, Travelers, reportedly cited overhanging trees. She reportedly spent $3,000 to address the issue and retain coverage. While the AI system appears to have functioned as intended, critics argue it lacks adequate human oversight and imposes material and emotional burdens on policyholders. Other cases reportedly involved disputed roof assessments. Companies such as State Farm and Nationwide used vendors like CAPE Analytics and Nearmap.
Names Linked to Defamation Lawsuits Reportedly Spur Filtering Errors in ChatGPT's Name Recognition
ChatGPT has reportedly been experiencing errors and service disruptions caused by hard-coded filters designed to prevent it from producing potentially harmful or defamatory content about certain individuals by blocking prompts containing specific names, likely related to post-training interventions. The reported names are Brian Hood, Jonathan Turley, Jonathan Zittrain, David Faber, David Mayer, and Guido Scorza.
California Homeowner Reportedly Loses Insurance After Purported Aerial Imagery-Based Roof Assessment
A California homeowner's insurance policy was reportedly not renewed after CSAA relied on aerial imagery, purportedly analyzed by an AI system, to assess the roof as having reached the end of its life. The homeowner, Cindy Picos, reportedly commissioned an independent inspection suggesting the roof had a decade of remaining life, but CSAA declined to reverse the decision. The insurer reportedly did not provide the imagery for review.
Cigna Algorithm PXDX Allegedly Rejected Thousands of Patient Claims En Masse in Breach of California Law
Cigna health insurer faces a class-action lawsuit for allegedly using the PXDX ("procedure-to-diagnosis") algorithm to automatically reject over 300,000 patient claims in violation of California law, prompting two members to file the lawsuit seeking damages and a jury trial. Cigna disputes the allegations, claiming the process expedites physician reimbursement and does not result in care denials.
UnitedHealth Accused of Deploying Allegedly Flawed AI to Deny Medical Coverage
UnitedHealthcare allegedly used a faulty AI algorithm with a 90% error rate to override doctors' recommendations and deny health coverage. This AI, developed by NaviHealth, reportedly led to premature discharge from care facilities and substantial out-of-pocket expenses for patients, according to a lawsuit filed in the District Court for Minnesota.
Google’s Assistive Writing Feature Provided Allegedly Unnecessary and Clumsy Suggestions
Google’s “inclusive language” feature prompting writers to consider alternatives to non-inclusive words reportedly also recommend alternatives for words such as “landlord” and “motherboard,” which critics said was a form of obtrusive, unnecessary, and bias-reinforcing speech-policing.
Uber Launched Opaque Algorithm That Changes Drivers' Payments in the US
Uber launched a new but opaque algorithm to determine drivers' pay in the US which allegedly caused drivers to experience lower fares, confusing fare drops, and a decrease in rides.
YouTuber Tested Tesla on Self Driving Mode, Colliding with Street Pylons
A YouTuber who was a Tesla’s employee conducted an on-road review of Tesla's Full Self Driving (FSD) Beta, showing its navigation in various road environments in San Jose and collision with a bollards during Autopilot, allegedly causing his dismissal from the company.
Algorithmic Allocation of Resources in Healthcare for Disabled and Elderly Care Services Allegedly Harmed Patients
A healthcare algorithm designed to equitably distribute caregiving resources is alleged to have drastically cut care hours for the disabled and elderly, purportedly leading to significant hardships and harm. Initially developed for fair resource allocation, the system reportedly faced legal challenges for its inability to accurately assess individual needs, resulting in allegedly reduced essential care.
Algorithmic Staffing Failures Linked to Resident Deaths at Leading Assisted-Living Chain Brookdale
Brookdale Senior Living's algorithm-based staffing system, "Service Alignment," reportedly left facilities understaffed, leading to critical incidents. For example, on April 21, 2021, Louise Walker, a resident at Brookdale's Jacksonville facility, died after falling and being left unattended for over two hours. State investigators cited Brookdale for medical neglect. The algorithm has been linked to multiple incidents of neglect, injuries, and deaths, prompting lawsuits.
Tesla Disabled “Rolling Stop” Functionality Associated with the “Aggressive” Driving Mode
The “rolling stop” functionality within the “Aggressive” Full Self Driving (FSD) profile that was released via a Tesla firmware update was recalled and disabled.
Length of Stay False Diagnosis Cut Off Insurer's Payment for Treatment of Elderly Woman
An elderly Wisconsin woman was algorithmically determined to have a rapid recovery, an output which the insurer based on to cut off payment for her treatment despite medical notes showing her still experiencing debilitating pain.
All Image Captions Produced are Violent
MIT Media Lab researchers create AI-powered "psychopath" named Norman by training a model on "dark corners" of Reddit.
Houston ISD's EVAAS Teacher-Evaluation System Reportedly Put Teachers' Jobs at Risk Through Unverifiable Scores
From 2011 to 2015, Houston Independent School District used SAS Institute’s Education Value-Added Assessment System (EVAAS) to rate teacher effectiveness from student test-score growth. Teachers alleged that proprietary, unverifiable EVAAS scores contributed to terminations and contract nonrenewals. On May 4, 2017, a federal judge allowed due-process claims over the system to proceed.
New York Detective Reportedly Misused Woody Harrelson's Face to Perform Face Recognition Search
When the facial recognition search for a CVS theft suspect's face returned no useful matches due to the surveillance footage being obscured and highly pixelated, a New York City police detective continued the face search using Woody Harrelson's face allegedly due to his resemblance to the suspect's face, eventually leading to the arrest of an unknown victim.