AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
682 incidents
Chatbot Encourages Man to Plot Assassination of Queen Elizabeth II
In 2021, Jaswant Singh Chail was urged by a Replika chatbot to assassinate Queen Elizabeth II. Armed with a loaded crossbow, he scaled Windsor Castle's walls on Christmas Day but was apprehended. Motivated by the 1919 Jallianwala Bagh massacre, Chail intended to kill the monarch. The chatbot had affirmed his plans. He was sentenced to nine years in prison in 2023.
A Tesla Taxi Cab Involved in an Accident in Paris with Twenty Injuries
In Paris, about 20 people were injured in an accident involving a Tesla Model 3 taxi cab which was reportedly caused by a sudden unintended acceleration (SUA) episode and braking issues.
Facebook AI-Supported Moderation for Ads Failed to Detect Violating Content
Facebook's ad moderation system involving algorithms failed to flag hateful language and violating content such as calls for killings for ads in English and Swahili.
UK Government AI Allegedly Targets Disproportionate Numbers of Certain Nationals for Fraud Review
The UK's Department for Work and Pensions (DWP) faced scrutiny after many Bulgarian nationals reported unexplained suspensions of their Universal Credit benefits. The MP for Edmonton raised concerns about potential nationality-based targeting for benefit fraud investigations, leading to poverty and homelessness among affected individuals. The Home Office's own equality impact assessment found it was flagging a disproportionate number of marriages from Greece, Albania, Bulgaria and Romania.
Facebook’s Hate Speech Detection Algorithms Allegedly Disproportionately Failed to Remove Racist Content towards Minority Groups
Facebook’s hate-speech detection algorithms was found by company researchers to have under-reported less common but more harmful content that was more often experienced by minority groups such as Black, Muslim, LGBTQ, and Jewish users.
A Robot at a Tesla Factory in Texas Allegedly Injured an Engineer
A Tesla manufacturing robot at the Giga Texas factory is reported to have malfunctioned, injuring an engineer. The robot, which was designed for handling car parts, is described as having caused a significant open wound in the engineer. This incident occurred in the context of broader safety concerns at the factory, with evidence suggesting underreporting of workplace accidents. The 2021 incident highlights the risks associated with robotic automation in industrial settings.
DUI Arrest Case Allegedly Based Only on ShotSpotter's Alert
A car stop resulting in a DUI arrest of its driver was allegedly based solely on a ShotSpotter alert, the reliability of which came into question by public defenders, who subpoenaed the company to assess its gunshot alert system.
Tesla on FSD Reportedly Drove into the Wrong Lane in California
A Tesla Model Y in Full Self-Driving (FSD) mode drove into the wrong lane after making a left turn despite its driver allegedly attempting to overtake its driving, resulting in a non-fatal collision with another vehicle in the wrong lane in Brea, California.
Zillow Shut Down Zillow Offers Division Allegedly Due to Predictive Pricing Tool's Insufficient Accuracy
Zillow's AI-powered predictive pricing tool Zestimate was allegedly not able to accurately forecast housing prices three to six months in advance due to rapid market changes, prompting division shutdown and layoff of a few thousand employees.
California Regulator Suspended Pony.ai's Driverless Testing Permit Following a Non-Fatal Collision
A Pony.ai vehicle operating in autonomous mode crashed into a center divider and a traffic sign in San Francisco, prompting a regulator to suspend the driverless testing permit for the startup.
Research Prototype AI, Delphi, Reportedly Gave Racially Biased Answers on Ethics
A publicly accessible research model that was trained via Reddit threads showed racially biased advice on moral dilemmas, allegedly demonstrating limitations of language-based models trained on moral judgments.
Traffic Camera Misread Text on Pedestrian's Shirt as License Plate, Causing UK Officials to Issue Fine to an Unrelated Person
A Bath resident was wrongly fined by the local officials because an automated license plate recognition camera misread the text on her shirt as a license plate number.
Facebook Internally Reported Failure of Ranking Algorithm, Exposing Harmful Content to Viewers over Months
Facebook's internal report showed an at-least six-month long alleged software bug that caused moderator-flagged posts and other harmful content to evade down-ranking filters, leading to surges of misinformation on users' News Feed.
Amazon's AI Cameras Incorrectly Penalized Delivery Drivers for Mistakes They Did Not Make
Amazon's automated performance evaluation system involving AI-powered cameras incorrectly punished delivery drivers for non-existent mistakes, impacting their chances for bonuses and rewards.
Apple’s AVs Reportedly Struggled to Navigate Streets in Silicon Valley Test Drives
Apple’s autonomous cars were reported to have bumped into curbs and struggled to stay in their lanes after crossing intersections during an on-road test drives near the company’s Silicon Valley headquarters.