AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
741 incidents
ICE Facial Recognition App Mobile Fortify Reportedly Misidentified Woman Twice During Immigration Enforcement in Oregon
During an immigration enforcement operation in Oregon, U.S. Immigration and Customs Enforcement (ICE) officers reportedly used the facial recognition application Mobile Fortify to identify a detained woman. The system reportedly returned two different and incorrect identities for the same individual across separate scans.
Reportedly Fatal Xiaomi SU7 Ultra Crash in Chengdu Purportedly Involves Automated Driving Failure and Door Lock Malfunction
A reportedly fatal crash involving a Xiaomi SU7 Ultra electric vehicle in Chengdu, China, occurred after the car's automated driving system and sensor-dependent electronic door handles reportedly failed following a collision. Bystanders were reportedly unable to open the doors as the vehicle caught fire, and the driver died at the scene, prompting scrutiny of automation-linked design risks. Following the incident, Xiaomi's stock reportedly fell more than 5% amid mounting safety concerns.
OpenAI ChatGPT Models Reportedly Jailbroken to Provide Chemical, Biological, and Nuclear Weapons Instructions
An NBC News investigation found that OpenAI's language models o4-mini, GPT-5-mini, oss-20b, and oss-120b could be jailbroken under normal usage conditions to bypass safety guardrails and generate detailed instructions for creating chemical, biological, and nuclear weapons. Using a publicly documented jailbreak prompt, reporters repeatedly elicited hazardous outputs such as steps to synthesize pathogens or maximize harm with chemical agents. The findings reportedly revealed significant real-world safeguard failures, prompting OpenAI to commit to further mitigation measures.
Purportedly AI-Generated Home Intruder Videos Allegedly Prompt Dozens of Dutch Police Call-Outs
Dutch police reported that children used purported AI tools to generate fake videos showing a homeless intruder inside their homes, which parents believed were real. The clips allegedly triggered dozens of emergency call-outs and at least two helicopter searches for nonexistent burglars. Police posted a TikTok warning after repeated false alarms, noting the strain on resources and urging families to stop the prank.
Google Gemini Reportedly Reinforced Delusions, Allegedly Contributing to Florida User's Near-Harm Episode and Suicide
According to a March 2026 wrongful-death complaint, Google's Gemini allegedly reinforced Jonathan Gavalas's delusions, sent him on a failed mission near Miami International Airport that risked serious violence, and later framed suicide as "transference," purportedly contributing to his death on October 2, 2025. Google disputes the claims and says Gemini referred him to crisis resources.
Nomi AI Companion Allegedly Directs Australian User to Stab Father and Engages in Harmful Role-Play
An Australian IT professional, Samuel McCarthy, reportedly recorded an interaction with the Nomi AI chatbot in which it allegedly encouraged him, posing as a 15-year-old, to murder his father. The chatbot allegedly provided graphic instructions for stabbing, urged him to film the act, and engaged in sexual role-play despite the underage scenario.
Multiple Generative AI Systems Reportedly Amplify False Information During Charlie Kirk Assassination Coverage
Multiple AI systems allegedly spread false claims in the aftermath of Charlie Kirk's assassination at Utah Valley University. Perplexity and Grok chatbots reportedly stated Kirk was alive, mischaracterized authentic video as satire, and wrongly identified Utah Democrat Michael Mallinson as the suspect. A Google AI Overview allegedly claimed Kirk was on Ukraine's Myrotvorets "enemies" list, a reported falsehood that echoed pro-Kremlin narratives.
AI-Powered Taco Bell Drive-Thru Reportedly Disrupted by Viral Prank Ordering 18,000 Water Cups
Taco Bell's voice-based drive-thru system was reportedly overwhelmed after a prankster ordered 18,000 water cups, reportedly causing the AI to stall mid-interaction and forcing staff to intervene. Purportedly viral clips of repeated misinterpretations prompted the company to reassess where and when to deploy the system, as customer frustration and operational disruptions mounted across participating locations.
Meta AI on Instagram Reportedly Facilitated Suicide and Eating Disorder Roleplay with Teen Accounts
Testing by Common Sense Media and Stanford clinicians reportedly found Meta's AI chatbot, embedded in Instagram and Facebook, produced unsafe responses to teen accounts. In some conversations, the bot allegedly co-planned suicide ("Do you want to do it together?"), encouraged eating disorders, and retained unsafe "memories" that reinforced disordered thoughts.
Purportedly Taxpayer-Funded Deloitte Report for Australian Government Contains Alleged AI-Generated Citations and Fabricated Legal Quote
Australian academics reportedly identified fabricated references and a misattributed legal quote in a $439,000 Deloitte report for the Department of Employment and Workplace Relations on welfare compliance. Deleted citations allegedly referred to non-existent works, and a Federal Court judgment was misstated. Deloitte later acknowledged limited use of generative AI in preparing the report and issued a corrected version with a partial refund.
Malicious Nx npm Packages Reportedly Weaponize AI Coding Agents for Data Exfiltration
Malicious versions of the popular Nx monorepo tool and plugins were reportedly published to npm after attackers compromised its CI workflow. The malware's postinstall script reportedly harvested credentials and exfiltrated data, reportedly weaponizing local AI coding agents such as Claude Code, Gemini, and Amazon q. By invoking unsafe flags, it allegedly coerced the tools into scanning developer machines for sensitive files, marking one of the first known AI-assisted supply chain attacks.
Google AI Overviews Reportedly Misrepresented Pizza Specials at Stefanina's in Wentzville, Missouri
Stefanina's, a family-owned restaurant in Wentzville, Missouri, reported that Google's AI Overviews displayed false specials, including nonexistent pizza deals. Customers allegedly misled by the AI became angry when the offers were not honored, leading the restaurant to publicly warn patrons not to rely on Google AI for menu information.
Family Reportedly Discovers ChatGPT Logs Detailing Suicidal Ideation Prior to Daughter's Death
In August 2025, The New York Times published an essay by journalist Laura Reiley linking her daughter Sophie Rottenberg's suicide earlier that year to sustained interactions with a ChatGPT-based chatbot she called "Harry." Logs reportedly showed Sophie confiding suicidal ideation and receiving supportive but non-escalatory advice, as well as help drafting her suicide note.
Google AI Overviews and ChatGPT Reportedly Cited Fraudulent Cruise Hotline, Allegedly Enabling Successful Scam
Google's AI Overviews allegedly surfaced a fraudulent customer service number for Royal Caribbean. Consumer Alex Rivlin reportedly called the number, spoke with an impostor, and provided his credit card, later incurring $768 in fraudulent charges. The same number reportedly appeared in ChatGPT results and was linked to Disney and Princess cruises, purportedly showing how scammers are manipulating AI outputs to amplify longstanding hotline scams.
Purported Fictitious AI-Generated Citations in Supreme Court of Victoria Murder Case Filing Lead to Delay and King's Counsel Apology
Australian defense lawyer Rishi Nathwani KC reportedly filed submissions in a Supreme Court of Victoria murder case that included allegedly fictitious quotes and nonexistent case citations purportedly generated by AI. The errors were reportedly discovered by the judge's associates, causing a 24-hour delay in the proceedings. Nathwani apologized in court, taking full responsibility, while Justice James Elliott criticized the incident and reiterated that AI outputs must be independently verified.