AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
136 incidents
Anysphere AI Support Bot for Cursor Reportedly Invents Login Policy, Leading to Subscription Cancellations
In April 2025, users of Cursor, an AI-powered coding assistant developed by Anysphere, reported being logged out unexpectedly. An AI-powered support bot, "Sam," allegedly responded with an invented login policy to justify the behavior. The hallucinated policy was not based on any real company change. The incident reportedly led to subscription cancellations.
Springer Nature Book 'Mastering Machine Learning: From Basics to Advanced' Reportedly Published With Numerous Purportedly Nonexistent or Incorrect Citations
"Mastering Machine Learning: From Basics to Advanced," published by Springer Nature in April 2025, reportedly contained numerous purportedly nonexistent or materially incorrect academic citations. Independent checks allegedly found that many referenced works did not exist or were misattributed, with multiple named researchers reportedly confirming they did not author the cited material. The pattern of errors has been described as consistent with known LLM citation hallucinations.
Reported Emergence of 'Vegetative Electron Microscopy' in Scientific Papers Traced to Purported AI Training Data Contamination
Researchers reportedly traced the appearance of the nonsensical phrase "vegetative electron microscopy" in scientific papers to contamination in AI training data. Testing indicated that large language models such as GPT-3, GPT-4, and Claude 3.5 may reproduce the term. The error allegedly originated from a digitization mistake that merged unrelated words during scanning, and a later translation error between Farsi and English.
$31,000 Sanction in Lacey v. State Farm Tied to Purportedly Undisclosed Use of LLMs and Erroneous Citations
In the case of Lacey v. State Farm, two law firms were sanctioned $31,000 after submitting a legal brief containing reportedly erroneous citations generated using AI tools. The court reportedly found that the lawyers failed to disclose the use of AI, neglected to verify its output, and refiled a revised brief with additional inaccuracies. Judge Michael Wilner deemed the conduct reckless and issued sanctions for what he described as "improper" and "misleading" legal filings.
Norwegian Supreme Court Receives Legal Filing with Fabricated Citations Allegedly Generated by AI Tool
In early 2025, Norway’s Supreme Court (Høyesterett) received a legal filing containing hallucinated legal citations and quotes, allegedly inserted by an attorney using an AI tool without verification. The fabricated sources were detected during standard review. While no sanctions were issued, the Court updated its guidance to lawyers, warning that AI use does not exempt them from professional responsibility.
AI-Driven News Platform Accused of Spreading Unverified Terrorism Allegations Against Yale Scholar
An AI-powered news site, Jewish Onliner, allegedly published an article linking Yale Law scholar Helyeh Doutaghi to Samidoun, a sanctioned group, which led to her suspension within days. The site, which claims to use AI for research, fact-checking, and content creation, did not disclose human reporters. Doutaghi denied the allegations and described the move as an attack on academic freedom.
Purportedly AI-Assisted Report by Tromsø Officials Allegedly Cited Non-Existent Sources in School Closure Proposal
In February 2025, Tromsø Municipality reportedly presented a report proposing the closure of schools and kindergartens. Residents reportedly later discovered the report cited numerous non-existent sources. Officials reportedly admitted using generative AI to help draft the report, leading to fabricated references. The incident reportedly led to public outcry and suspension of the closure plan. It is purportedly Norway's first major public-sector AI scandal.
NHK Terminates AI Translation Service After Geopolitical Naming Error
NHK announced the termination of its AI-powered multilingual subtitle service after an automatic translation error rendered "Senkaku Islands" as "Diaoyu Islands," the Chinese designation. The mistake, discovered on February 10th, 2025 during a news segment, led NHK to deem the service inappropriate. The AI-based subtitles, powered by Google Translate, had been in use since 2020.
Plaintiffs' Lawyers Admit AI Generated Erroneous Case Citations in Federal Court Filing Against Walmart
Lawyers Rudwin Ayala, T. Michael Morgan (Morgan & Morgan), and Taly Goody (Goody Law Group) were fined a total of $5,000 after their Wyoming federal lawsuit filing against Walmart cited fake cases "hallucinated" by AI. Judge Kelly Rankin sanctioned them, removing Ayala from the case and noting attorneys must verify AI sources. The filing, flagged by Walmart’s legal team, led to its withdrawal and an internal review.
AI-Generated Images of the Iconic Hollywood Sign Reportedly on Fire Circulating on Social Media
During the January 2025 Southern California wildfires, AI-generated images of the iconic Hollywood Sign being on fire circulated on social media.
Apple Intelligence Reportedly Notified Users That Luigi Mangione Shot Himself and Benjamin Netanyahu Had Been Arrested
Apple Intelligence reportedly sent push notifications falsely claiming that BBC News had reported Luigi Mangione's suicide and that The New York Times had reported Benjamin Netanyahu's arrest. On 1/16/2025, Apple reportedly disabled Apple Intelligence's notification summaries.
Alleged Fake Citations Undermine Expert Testimony in Minnesota Deepfake Law Case
In a legal case defending Minnesota’s deepfake election misinformation law, Stanford misinformation expert Professor Jeff Hancock's affidavit allegedly cited non-existent academic sources, potentially generated by ChatGPT. The reportedly fabricated citations appear to have undermined the credibility of his testimony.
AI Models Reportedly Found to Provide Misinformation on Election Processes in Spanish
An analysis reportedly found that multiple AI models provided inaccurate responses to election-related questions, with 52% of Spanish-language answers and 43% of English-language answers containing misinformation or omissions. Errors included misidentifying voting processes and providing information about foreign elections.
Alaska Education Department Reportedly Published Policy Featuring Erroneous AI-Generated Citations
In October 2024, Alaska’s Education Commissioner Deena Bishop allegedly used generative AI to draft a school cellphone policy, resulting in reportedly fabricated citations of non-existent studies. The document, which did not disclose AI use, was shared with the state Board of Education, and despite reported corrections, AI-generated errors remained in the final resolution.
AI Transcription Tool Whisper Reportedly Inserting Fabricated Content in Medical Transcripts
OpenAI's AI-powered transcription tool Whisper, used to translate and transcribe audio content such as patient consultations with doctors, is advertised as having near “human level robustness and accuracy.” However, software engineers, developers and academic researchers have alleged that it is prone to making up chunks of text or even entire sentences and that some of the hallucinations can include racial commentary, violent rhetoric, and even imagined medical treatments.