AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
736 incidents
Deepfake Nudes Targeting Underage Female Students at Collège Béliveau in Winnipeg Shared Online
At Collège Béliveau in Winnipeg, female students between grades 7-12 were targeted in the creation of deepfake nudes, which were then distributed online. Specific numbers and identities of victims and perpetrators were not released, and no charges were ultimately filed owing to the gap between existing laws and the nature of the incident.
Alleged Deepfake of Singapore Deputy Prime Minister Lawrence Wong Falsely Shows Him Endorsing Commercial Products
A deepfake video falsely depicting Singapore Deputy Prime Minister Lawrence Wong endorsing commercial products circulated online in December 2023. Wong publicly denied the endorsement, warning that scammers had used AI-generated deepfake technology to impersonate him. The manipulated media aimed to mislead the public into believing he promoted fraudulent investments. Wong urged vigilance against AI-driven scams and encouraged reporting through the ScamShield Bot on WhatsApp.
Students at a Beverly Hills Middle School Allegedly Created and Shared Deepfake Nudes of Their Classmates
At Beverly Vista Middle School in Beverly Hills, California, students allegedly used AI to generate fake nude photos with their classmates' faces, prompting investigations by school officials and the police. The incident highlights the increasing misuse of generative AI among minors.
Two Florida Middle School Students Arrested Under New Law for Allegedly Having Made and Shared Deepfake Nudes of Their Classmates
Two teenaged boys from Miami, Florida, were arrested for allegedly creating and sharing AI-generated nude images of their classmates. Charged under a 2022 Florida law, they face third-degree felonies for producing and disseminating altered sexual depictions.
Video with Reportedly AI-Generated Audio Purported to Show Croatian Footballer Luka Modrić Endorsing Immediate Matrix on Facebook Page Presented as N1 HR
A Facebook page presented as N1 HR, a Croatian news outlet, posted a video that purported to show Croatian footballer Luka Modrić endorsing the investment platform Immediate Matrix and promising returns of up to €4,000 per month. The clip was reported to use AI-generated audio and to link to an Index-style article promoting the scheme.
OpenDream AI Platform Reportedly Commercialized AI-Generated CSAM and Non-consensual Deepfake Sexual Images
OpenDream, an AI image generation platform operated by CBM Media Pte Ltd, reportedly allowed users to generate and monetize AI-generated CSAM and non-consensual sexual deepfakes through its public gallery and paid NSFW tools. The content was reported to be publicly accessible and indexed by search engines for months before limited moderation occurred in mid-2024, prompting takedowns by Google and Bing and the termination of its ad and payment services.
Underground Market for LLMs Powers Malware and Phishing Scams
A study by Indiana University researchers uncovered widespread misuse of large language models (LLMs) for cybercrime. Cybercriminals, according to that study, use LLMs like OpenAI's GPT-3.5 and GPT-4 to create malware, phishing scams, and scam websites. These models are available on underground markets, often bypassing safety checks through jailbreaking. Named malicious LLMs are BadGPT, XXXGPT, Evil-GPT, WormGPT, FraudGPT, BLACKHATGPT, EscapeGPT, DarkGPT, and WolfGPT.
Sports Illustrated Is Alleged to Have Used AI to Invent Fake Authors and Their Articles
Sports Illustrated, managed by The Arena Group, allegedly used AI-generated authors and content, compromising journalistic integrity. Profiles of these fictitious authors, complete with AI-generated headshots, appeared alongside articles, misleading readers. The issue was exposed when inconsistencies in author identities and writing quality were noticed, leading to the removal of this content from the publication's website.
Video Allegedly Altered by AI Reportedly Spreads Claim of Nigerian Doctor's Hypertension Cure
An alleged AI-manipulated video circulating on Facebook falsely claims that a Nigerian doctor has developed a drug that cures high blood pressure permanently within three days. The video, which reportedly mimics Channels Television news anchor Kayode Okikiolu, features manipulated audio and misleading content, including unverified guarantees of reducing stroke and heart attack risks to zero.
Deepfake Video Circulating of British Labour Leader Keir Starmer Touting an Investment Scheme
A deepfake video was circulating around social media of British Labour leader Keir Starmer touting an investment scheme.
Male student allegedly used AI to generate nude photos of female classmates at a high school in Issaquah, Washington
At a high school in Issaquah, Washington, a male student is reported to have used deepfake technology to alter pictures of several female classmates and then shared them.
Kremlin-Linked Entities Allegedly Using Generative AI to Spread Russian Disinformation in Latin America
Moscow-based tech firms and an industry association with links to the Kremlin are allegedly using generative AI to spread Russian disinformation in countries throughout Central America and South America. According to the U.S. Department of State, the Russian companies rely on local writers to compose stories which are then amplified across social media using artificial intelligence chatbots.
Alleged Misuse of PicSo AI for Generating Inappropriate Content Emphasizing "Girls"
PicSo AI, which appears to be getting advertised by Meta over Instagram, is allegedly being used for generating inappropriate content with an emphasis on "girls." This raises concerns about the misuse of generative AI technologies for creating offensive and potentially sexually explicit material that could be used for nefarious and criminal purposes.
Reported Use of AI Voice and Identity Manipulation in the Ongoing 'Phantom Hacker' Fraud Scheme
Reports allege that updated variants of the long-running "Phantom Hacker" scam use purported AI tools to enhance impersonation, including voice cloning, spoofed caller ID, and realistic digital artifacts. Fraudsters reportedly pose as tech support, bank staff, and government officials in a three-phase scheme that pressures mostly older adults to transfer funds to accounts controlled by scammers.
Female Students at Westfield High School in New Jersey Reportedly Targeted with Deepfake Nudes
In October 2023, alleged deepfake nude images of female students were created and shared at Westfield High School in New Jersey. One alleged implicated system in this incident is the Clothoff deepfake app. A victim, Francesca Mani, and her mother, Dorota, began advocating for stronger laws after the school’s response reportedly left perpetrators unaccountable. Since the incident, their advocacy has drawn national attention, leading to trips to Washington, D.C., and bipartisan backing for legislation targeting AI-generated nonconsensual content while calling attention to deficiencies in existing legal protections.