AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
196 incidents
Catholic AI Chatbot 'Father Justin' Claimed to Be a Real Priest, Prompting Retraction
Catholic advocacy group Catholic Answers released an AI priest called "Father Justin," which misleadingly claimed to be a real clergy member, offered sacraments, and provided controversial advice. After receiving criticism, the group rebranded the chatbot as a lay theologian to correct the misrepresentation. The incident is an instructive case with respect to deploying AI in sensitive contexts and the potential for causing confusion and harm.
Microsoft Copilot Allegedly Provides Unsafe Medical Advice with High Risk of Severe Harm
Microsoft Copilot, when asked medical questions, was reportedly found to provide accurate information only 54% of the time, according to European researchers (citation provided in editor's notes). Analysis by the researchers reported that 42% of Copilot's responses could cause moderate to severe harm, with 22% of responses posing a risk of death or severe injury.
Purported Error by Grok Reportedly Misrepresents Basketball Slang as Criminal Allegation Against NBA Player
X's Grok AI bot reportedly generated a trending story falsely claiming that Golden State Warriors guard Klay Thompson was under investigation for vandalizing homes with bricks in Sacramento. The alleged fabrication appeared after Grok reportedly misinterpreted posts describing Thompson "shooting bricks" during a basketball game. The story reportedly ranked #5 briefly on X's trending list.
Google Books Appears to Be Indexing Works Written by AI
Google Books is indexing low-quality, AI-generated books, degrading its database and potentially distorting Google Ngram Viewer's analysis of language trends. This integration of inaccurate or misleading information undermines trust, disseminates poor-quality content, and wastes resources as researchers must spend time clearing up the misinformation.
Grok AI on X Created and Promoted False Iran Missile Strike News
On April 4, 2024, X's AI chatbot Grok generated a false headline claiming "Iran Strikes Tel Aviv with Heavy Missiles," which was then promoted on X's trending news section. This misinformation, fueled by user spamming of fake news, falsely indicated a serious international conflict. The incident highlighted significant risks associated with relying on AI for content curation and demonstrated the potential for widespread dissemination of harmful misinformation.
AI Deepfakes for Voter Outreach Flood Indian Elections
During the 2024 Indian elections, politicians used AI-generated deepfakes to reach voters, who might be unaware they're interacting with digital clones. Providers like Divyendra Singh Jadoun of Polymath Synthetic Media Solutions created deepfakes for personalized messages. This practice, used by various political parties, is not truthful, as voters may be misled by AI-generated content posing as genuine interactions with political figures.
Microsoft-Powered New York City Chatbot Advises Illegal Practices
New York City's chatbot, launched under Mayor Eric Adams's plan to assist businesses, has been reportedly providing dangerously inaccurate legal advice. The Microsoft-powered bot allegedly informed users that landlords can refuse Section 8 vouchers and that businesses can operate cash-free, among other falsehoods. The city acknowledges the chatbot is a pilot program and commits to improvements while the errors are addressed.
The Arizona Agenda Produced a Deepfake of Kari Lake Advocating for the Publication Without Her Consent
The Arizona Agenda produced a deepfake video of Republican Senate candidate Kari Lake giving a testimonial about the publication with the seeming intention of educating the general public about the dangers of deepfakes in the coming election cycle. However, the Arizona Agenda appears not to have sought Lake's consent, prompting a cease-and-desist letter from her campaign.
Alleged Deepfake Video Depicts Former President of Kenya Uhuru Kenyatta Announcing 2027 Presidential Bid
A widely shared TikTok video purportedly manipulated by AI depicts former Kenyan president Uhuru Kenyatta announcing a 2027 presidential run. Reported digital forensics and visual inconsistencies confirm the video is an AI-generated deepfake, using archival footage from his presidency with synthesized audio. The clip spread rapidly online but has reportedly been debunked by fact-checkers. Kenyatta has made no such announcement, and such a move would violate Kenya's two-term constitutional limit.
Seeming Pattern of Gemini Bias and Sociotechnical Training Failures Harm Google's Reputation
Google's Gemini chatbot faced many reported bias issues upon release, leading to a variety of problematic outputs like racial inaccuracies and political biases, including regarding Chinese and Indian politics. It also reportedly over-corrected racial diversity in historical contexts and advanced controversial perspectives, prompting a temporary halt and an apology from Google.
Purported AI-Cloned Voice Depicts Kenyan President William Ruto Speaking French in Viral Video
A purportedly AI-manipulated video widely shared on TikTok and Facebook presents Kenyan President William Ruto speaking in French during a speech in Tokyo on February 7, 2024. A reported reverse image search confirmed the original footage shows Ruto speaking in English to the Kenyan diaspora. The reportedly manipulated version used AI-generated French audio over the original video, making it a case of voice cloning rather than a full audiovisual deepfake.
AI-Generated Fake News Targets Black Celebrities on YouTube
YouTube faced a surge of AI-generated fake news targeting Black celebrities, including fake narratives about Sean “Diddy” Combs and others. These videos, blending AI-generated and manipulated media, amassed millions of views, challenging content moderation efforts and highlighting the spread of disinformation.
Deepfakes of Deceased Indian Politicians for Election Campaigning Are Increasingly Being Deployed
In the lead-up to India's 2024 general elections, AI technology was used to create deepfake videos of deceased politicians, such as M. Karunanidhi and J. Jayalalithaa, aiming to influence voter behavior and campaign strategies. These AI-generated appearances are contributing to the erosion of trust in democratic processes and media discourse.
Purported Deepfake Audio Reportedly Sparks False Claims of Biden Threatening Texas with F-15s
A purportedly AI-generated audio clip imitating President Biden’s voice reportedly circulated on social media, falsely suggesting that he had threatened to deploy F-15 fighter jets against Texas. The manipulated recording was widely shared and is reported to have heightened political tensions and spread misinformation.
Proliferation of Products on Amazon Titled with ChatGPT Error Messages
Products named after ChatGPT error messages are proliferating on Amazon, such as lawn chairs and religious texts. These names, often resembling AI-generated errors, indicate a lack of editing and undermine the sense of authenticity and reliability of product listings.