AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
314 incidents
Reported Glitch in Waymo Self-Driving Cars Purportedly Triggers Regular All-Night Honking in San Francisco
Waymo self-driving cars in San Francisco's South of Market neighborhood reportedly began honking at each other late at night, disturbing residents' sleep. The autonomous vehicles, reportedly using a parking lot for ride pauses, triggered honking due to a purported glitch in their algorithms. Despite residents' complaints, the issue allegedly persisted for weeks until Waymo acknowledged the problem and began working on a fix.
Apple Intelligence Allegedly Flags Scam Messages as High Priority
Apple’s AI notification summarization and prioritization feature, part of the Apple Intelligence update, has reportedly flagged scam and phishing messages as high-priority, and is allegedly misleading users into trusting fraudulent communications. By rephrasing and condensing messages, the system is allegedly amplifying scam risks inadvertently and reportedly increasing the likelihood of financial harm for users.
Fatalities Reportedly Occur Despite VioGén Algorithm's Low or Negligible Risk Scores
The VioGén algorithm was designed to help Spanish police assess and prioritize the risk of repeat domestic violence incidents. However, its low-risk assessment of Lobna Hemid reportedly led to inadequate protection; her husband murdered her. Since 2007, 247 women have been killed after being assessed by VioGén. A review of 98 homicides found that 55 of the slain women were scored as negligible or low risk.
Baidu Robotaxi Allegedly Involved in Collision with Pedestrian in Wuhan
On July 7, 2024, a Baidu Robotaxi reportedly collided with a pedestrian at a traffic intersection in Wuhan. The incident occurred as the autonomous vehicle started moving on a green light while the pedestrian was allegedly crossing against a red light. The pedestrian sustained minor injuries, and Baidu was reported to have been cooperating with local authorities for further investigation.
Tesla Reportedly in Autopilot Mode Hits Parked Police Vehicle in Fullerton, California
A Tesla reportedly in self-driving mode crashed into a parked patrol vehicle in Fullerton, California while the officer was responding to a fatal DUI crash. The officer narrowly escaped injury. The driver reports having been distracted by a cellphone and having relied on the Tesla’s AI. (The earlier crash involved a suspected DUI driver who killed a motorcyclist stopped at a red light.)
Independent News Sites Flagged as Spam by Facebook's AI Moderation System
Facebook's AI moderation system mistakenly flagged and removed legitimate news articles from independent local publishers as spam. This affected publishers worldwide, disrupting content distribution and impacting traffic. Despite attempts to appeal, many publishers received no response.
Facebook's Content Moderation System Flagged and Removed Emergency Updates as Spam During Wildfires
Facebook's AI moderation system wrongly flagged and removed dozens of posts containing vital emergency information during California's wildfire season, including real-time updates on evacuations and fire tracking. Posts from official sources like Cal Fire and the U.S. Forest Service were marked as spam, potentially endangering lives by restricting access to crucial updates. Despite user complaints, the issue persisted, with Facebook acknowledging the problem only after media inquiry.
Class Action Lawsuit Over Alleged Defects in Volkswagen's AI-Driven AEB Systems
A class action lawsuit involving Volkswagen Group of America addresses alleged defects in the Automated Emergency Braking (AEB) systems of certain vehicles. The lawsuit claims these AI-driven systems failed to function properly, posing safety risks. Volkswagen denies the claims but has agreed to a settlement. Affected users can look up their vehicle's eligibility and file claims for reimbursement. The case brings into question the level of reliability of AI in critical automotive applications.
GPT-4o's Chinese Tokens Reportedly Compromised by Spam and Pornography Due to Inadequate Filtering
OpenAI's GPT-4o was found to have its Chinese token training data compromised by spam and pornographic phrases due to inadequate data cleaning. Tianle Cai, a Ph.D. student at Princeton University, identified that most of the longest Chinese tokens were irrelevant and inappropriate, primarily originating from spam and pornography websites. The polluted tokens could lead to hallucinations, poor performance, and potential misuse, undermining the chatbot's reliability and safety measures.
Google Healthcare AI Model Med‑Gemini Allegedly Produces Non‑Existent 'Basilar Ganglia' Term in Published Output
Google’s Med‑Gemini healthcare AI reportedly produced the non‑existent term "basilar ganglia" in public launch materials, conflating two distinct brain structures. The error reportedly appeared in both a blog post and an arXiv preprint. Google is reported to have initially edited the blog without acknowledgment, later calling it a typo.
Cybercheck Tool Allegedly Provides Questionable Evidence in Murder Trials
Global Intelligence's Cybercheck AI tool, used by law enforcement to track suspects based on open source data, has allegedly been providing inaccurate or unverifiable evidence in several murder trials. Reportedly the tool lacks transparency and often produces unreliable reports, which has prompted prosecutors to withdraw Cybercheck evidence from multiple cases after its findings were challenged, reportedly wasting law enforcement time and resources while undermining prosecutors' cases.
NHTSA Opens New Probe into Tesla’s Autopilot Following More than a Dozen Fatal Accidents
The NHTSA has linked Tesla's Autopilot to over a dozen fatalities and hundreds of crashes, prompting a new investigation into the adequacy of Tesla's December recall of 2 million vehicles. The probe reports that Tesla’s driver-assist system led to avoidable crashes involving visible hazards, suggesting a critical safety gap between driver expectations and the system’s capabilities. The investigation will assess if Tesla’s recall remedies were sufficient to address these safety risks.
The WHO's S.A.R.A.H. Bot Reported to Provide Inconsistent and Inadequate Health Information
The WHO's AI-powered health advisor, S.A.R.A.H. (Smart AI Resource Assistant for Health), is alleged to provide inconsistent and inadequate health information. The bot reportedly gives contradictory responses to the same queries, fails to offer specific contact details for healthcare providers, and inadequately handles severe mental health crises, often giving irrelevant or unhelpful advice.
Facebook AI Reportedly Mislabels Auschwitz Photos as 'Bullying' and 'Nudity'
Facebook's AI system allegedly mislabeled around 20 posts from the Auschwitz Memorial Museum as violating community standards for "bullying" and "nudity," reportedly deleting at least one image of orphans. The purported misclassification of historical content allegedly prompted outrage from the museum, which reportedly demanded an explanation. Meta later purportedly apologized, attributing the issue to mistaken notices sent by its AI system and reportedly acknowledged that the posts did not, in fact, violate company policies.
Brake Failure in Reportedly AI-Driven Tram Leads to Multiple Injuries in Saint Petersburg, Russia
During a test run in Saint Petersburg in Russia a reportedly AI-powered "Smart Tram" failed, leading to a crash that injured several pedestrians and trapped a woman under its wheels. The tram's AI system reportedly shut off unexpectedly, and both primary and emergency brakes failed despite pre-test checks.