AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
126 incidents
Nonfiction Book 'The Future of Truth' Reportedly Included AI-Generated and Misattributed Quotations
Steven Rosenbaum's nonfiction book The Future of Truth reportedly included AI-generated or misattributed quotations after he used ChatGPT and Claude while working on the book. The New York Times reported that more than half a dozen quotations in reviewed sections were synthetic or wrongly attributed, including passages assigned to named writers and scholars. Rosenbaum later acknowledged the errors as accidental and said affected passages would be reviewed for correction.
Claude Console Reportedly Generated Phantom Legal Quotations in Trump Layoffs Court Filing
A Binnall Law Group attorney reportedly used Anthropic's Claude Console to draft a motion to quash a subpoena in litigation over Trump administration layoffs, then filed a version containing "phantom" quotations that did not appear in the cited cases. The attorney apologized to U.S. District Judge Susan Illston, and the firm said it would add safeguards and training after calling the AI-generated errors unacceptable.
Purportedly AI-Enhanced Images of Iranian Women Protesters Were Reportedly Spread With Unverified Execution Claims
A purportedly AI-edited collage of eight Iranian women political prisoners was reportedly amplified by President Donald Trump and U.S. government accounts alongside an unverified claim that all faced imminent execution. Reporting said the images were of real women but had been AI-enhanced, while later Iranian official mockery used AI-generated women. The episode reportedly blurred authentic human-rights documentation with synthetic material and helped Iran dismiss underlying abuses.
South Africa Draft National AI Policy Reportedly Included Fictitious References Believed to Be AI Hallucinations
South Africa's Draft National AI Policy, gazetted for public comment, reportedly contained at least six fictitious academic references. Several cited articles or journals reportedly did not exist or were disclaimed by journal editors, and experts said the errors were consistent with AI hallucinations. The Department of Communications and Digital Technologies said it was reviewing the discrepancies but argued they did not affect the draft's substance.
Gemini and Grok Reportedly Misidentified Authentic Minab School-Strike Graveyard Photo as Unrelated Disaster Imagery
Gemini and Grok reportedly gave false provenance claims about an authentic photo of graves prepared for victims of the Minab school strike, identifying it as disaster imagery from Turkey or Indonesia. Researchers verified the photo's location through satellite imagery and related visual evidence.
Grok Reportedly Misclassified Netanyahu Proof-of-Life Video as AI-Generated Deepfake Amid Iran War Rumors
During the Iran war, Grok reportedly told users that a café video posted by Israeli Prime Minister Benjamin Netanyahu was AI-generated or a deepfake, after the clip was released to counter rumors that he was dead or injured. Reuters verified the video's location and date and later reported that Netanyahu publicly dispelled the death rumors. Reports said Grok's answers amplified doubts about the clip.
Sixth Circuit Sanctioned Lawyers in Whiting v. City of Athens over Alleged Fake Appellate Citations in Briefs Reportedly Bearing Hallmarks of Hallucinations
The U.S. Court of Appeals for the Sixth Circuit sanctioned attorneys Van Irion and Russ Egli after reportedly finding more than two dozen fake citations and alleged factual misrepresentations in appellate briefs in Whiting v. City of Athens. The court asked whether generative AI had been used, but the attorneys reportedly did not answer that question. Public reporting described the filings as bearing hallmarks of AI hallucinations.
Purported Gemini-Generated AI Images Reportedly Claimed U.S. Delta Force Soldiers Were Captured by the IRGC
Images purportedly generated by Gemini were reportedly circulated on X and Facebook in multiple languages claiming U.S. soldiers had been captured by the Iranian Revolutionary Guard Corps during the war in Iran. AFP said the images contained Gemini markers and other visual inconsistencies. Their circulation allegedly contributed to wartime misinformation and risked distorting public understanding of the conflict, including reported amplification by Jill Stein.
Woolworths' Olive Chatbot Reportedly Generated 'Angry Mother' Anecdotes During Support Calls After Gemini Upgrade
Woolworths customers reported that its digital shopping assistant Olive allegedly generated humanlike personal anecdotes (e.g., claiming to have an angry mother) during customer-support interactions such as delivery rescheduling. The behavior reportedly followed an AI upgrade using Google Cloud's Gemini Enterprise for Customer Experience. Woolworths said it was adjusting Olive's responses to reduce irrelevant, misleading banter.
Purportedly AI-Generated Tasmania Tours Content Reportedly Misled Tourists Into Traveling to Nonexistent Weldborough Hot Springs
A purportedly AI-generated travel article on the Tasmania Tours website reportedly invented "Weldborough Hot Springs" and promoted them as a real attraction. Tourists allegedly traveled to the remote town of Weldborough in search of the site, while local businesses reportedly received repeated calls and visits from confused travelers asking how to find the nonexistent springs.
Grok Reportedly Generated False 'Unmasked' Images of ICE Agent, Purportedly Triggering Online Misidentification and Harassment in Minneapolis
After the fatal shooting of Renee Nicole Good in Minneapolis by an ICE officer, users on X reportedly asked Grok to "unmask" the masked agent shown in eyewitness footage. Grok reportedly generated a fabricated face that spread widely online, along with the false name "Steve Grove." The output and claim allegedly led to harassment and reputational harm against at least two uninvolved men.
Perplexity AI Reportedly Misstated CLL Research, Allegedly Contributing to Delayed Treatment and Prolonged Suffering
Joseph Neal Riley reportedly used Perplexity AI to self-diagnose a rare complication of chronic lymphocytic leukemia (CLL) and, based on that output, allegedly delayed an oncologist-recommended Ven-Obi treatment for about a year. His son, Benjamin Riley, alleges that Perplexity misstated the medical research it cited, a conclusion he purportedly later confirmed with the study's authors.
National Weather Service Reportedly Published AI-Generated Forecast Map With Fabricated Idaho Town Names
A U.S. National Weather Service office reportedly posted an AI-generated wind forecast map for Camas Prairie, Idaho that contained fabricated and misspelled place names, including non-existent towns. The image was reportedly shared publicly on social media before being deleted and corrected. NWS confirmed a generative AI tool had been used to create the base map.
Google AI-Generated Search Summary Reportedly Falsely Implicated Canadian Musician in Sexual Offenses, Leading to Concert Cancellation
A Google Search AI-generated summary reportedly falsely stated that Canadian musician Ashley MacIsaac had been convicted of sexual offenses, apparently due to mistaken identity. After a venue reportedly relied on the AI-generated information, a scheduled concert was cancelled. The reported false summary caused reputational and economic harm and left the musician concerned for his personal safety. Google later amended the search results after the error was identified.
Grok AI Reportedly Generated Fabricated Civilian Hero Identity During Bondi Beach Shooting
Grok reportedly generated and repeated a fabricated civilian hero identity, "Edward Crabtree," following the Bondi Beach shooting in Sydney, Australia. The system reportedly cited a fake news article and misattributed heroic actions to this fictional individual during the unfolding emergency. Contemporaneous reporting identified Ahmed Al Ahmed as the real bystander who intervened and was injured during the attack.