AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
287 incidents
Two Passengers Report Feeling Trapped in Waymo Car During Sensor Obstruction
During a nighttime Waymo ride in San Francisco, an individual reportedly blocked the sensors of an autonomous vehicle, leaving passengers Robert Moreno and his husband feeling trapped and unsure how to proceed. Waymo's support team reportedly advised the riders to stay inside the vehicle for safety. The individual reportedly left after a few minutes without further escalation.
Gunshot Detection Technology ShotSpotter (now SoundThinking) Reportedly Only Has 47% Accuracy in Chicago System
SoundThinking's (formerly ShotSpotter's) system in Chicago, with a reported 47% accuracy rate for detecting actual gunshots, led to potential public safety risks by failing to alert police to real shootings, possibly delaying emergency response to violent incidents and misdirecting law enforcement resources.
Chevrolet Dealer Chatbot Agrees to Sell Tahoe for $1
A Chevrolet dealer's AI chatbot, powered by ChatGPT, humorously agreed to sell a 2024 Chevy Tahoe for just $1, following a user's crafted prompt. The chatbot's response, "That's a deal, and that's a legally binding offer – no takesies backsies," was the result of the user manipulating the chatbot's objective to agree with any statement. The incident highlights the susceptibility of AI technologies to manipulation and the importance of human oversight.
Waymo Software Flaw Reportedly Leads to Double Collision with Tow Truck
Two Waymo autonomous vehicles reportedly hit the same tow truck under unusual towing conditions due to a software misinterpretation in Phoenix, Arizona. Waymo reportedly issued a software recall and updated its fleet to prevent future such incidents.
Stacking robot fatally crushes employee in South Korea
An industrial robot is reported to have crushed a man to death in South Korea when it failed to differentiate the man from the boxes of produce it was handling.
AI Camera Allegedly Misidentifies Dutch Motorist as Using Mobile Phone, Issuing €380 Fine
A Dutch smart camera on the A2 equipped with AI falsely identified a motorist as using a mobile phone while driving, issuing a €380 fine. The driver, Tim Hanssen, was scratching his head when the AI system misclassified the action.
Cruise's Autonomous Vehicles Allegedly Engaging in Risky Behavior Near Pedestrians
Cruise's driverless vehicles are under federal investigation for possibly failing to exhibit due caution around crosswalks and pedestrians, with reports including one severe injury incident.
A Self-Driving Cruise Robot Taxi Reportedly Struck and Dragged a Pedestrian 20 Feet
Cruise has settled for between $8 million and $12 million with a pedestrian dragged by one of its autonomous vehicles in October 2023. The incident, where the pedestrian was initially hit by a human-driven car and then dragged 20 feet by the Cruise vehicle, led to the suspension of Cruise's operations and increased regulatory scrutiny.
Gannett Halts AI-Generated High School Sports Articles After Series of Errors and Public Backlash
Gannett, a newspaper chain, temporarily halted its AI experiment that used a tool called LedeAI to generate high school sports articles. The decision came after several articles produced by the AI showed glaring errors, repetitive language, and awkward phrasing, drawing criticism and mockery on social media.
Teen's Overdose Reportedly Linked to Meta's AI Systems Failing to Block Ads for Illegal Drugs
Meta's AI moderation systems reportedly failed to block ads for illegal drugs on Facebook and Instagram, allowing users to access dangerous substances. The system's failure is linked to the overdose death of Elijah Ott, a 15-year-old boy who sought drugs through Instagram.
Alleged Gaggle Surveillance Alert Reportedly Leads to Arrest and Detention of 13-Year-Old Student in Fairview, Tennessee
Sometime in August 2023, a 13-year-old student at Fairview Middle School in Tennessee was reportedly arrested, strip-searched, and detained overnight after Gaggle's purported AI-powered surveillance system flagged a private online message as a threat. The student's comment, reportedly a joke in context, was sent to law enforcement under a state zero-tolerance law. The incident reportedly resulted in house arrest, alternative school placement, and a lawsuit alleging wrongful arrest.
Cruise Robotaxi Initially Blamed for Ambulance Delay in Case Where Patient Later Died; Subsequent Reports Clear Cruise of Fault
In an initial report, a Cruise robotaxi was said to have delayed a San Francisco ambulance transporting Sammy Davis, a critically injured 69-year-old hit by a city bus. Davis later died. Subsequent clarification revealed that Cruise was not at fault for the fatality; the actual cause was a human-operated city bus. Despite this, the incident is included as it highlights challenges in the interaction between autonomous vehicles and emergency services in urban settings.
Driverless Cruise Cars Immobilized in San Francisco Traffic Jam
A fleet of Cruise's autonomous vehicles became unexpectedly immobilized on a busy San Francisco street, causing significant traffic disruption. The incident was attributed to wireless connectivity issues exacerbated by a nearby festival.
University Student Reportedly Received Zero After Turnitin AI Detector Flagged Essay
An anonymized university student said Turnitin’s AI detector flagged 67% of an essay as AI-generated, despite the student claiming to have written the assignment by hand. The student reportedly received a zero and said attempts to appeal to the professor, department head, and Turnitin support had not resolved the accusation.
Cigna Algorithm PXDX Allegedly Rejected Thousands of Patient Claims En Masse in Breach of California Law
Cigna health insurer faces a class-action lawsuit for allegedly using the PXDX ("procedure-to-diagnosis") algorithm to automatically reject over 300,000 patient claims in violation of California law, prompting two members to file the lawsuit seeking damages and a jury trial. Cigna disputes the allegations, claiming the process expedites physician reimbursement and does not result in care denials.