AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
566 incidents
Mumbai Businessman Reportedly Defrauded via Purported AI-Cloned Voice Impersonating Son
A Mumbai businessman reportedly identified as KT Vinod reportedly lost Rs 80,000 after receiving a call from someone claiming to be a representative of the Indian Embassy in Dubai, who said his son had been arrested. The caller allegedly used AI-generated voice cloning to simulate the voice of Vinod's son, pleading for help. Reportedly convinced of the urgency, Vinod instructed a money transfer via Google Pay. The scam was discovered only after he contacted his son directly.
Scammers Using Deepfakes of Women's Faces and Voices for False and Offensive Advertisements
Scammers used AI tools from HeyGen and ElevenLabs to create deepfake videos of influencers Michel Janse, Olga Loiek, Shadé Zahrai, and Carrie Williams, misusing Lana Smalls's voice in Williams's case. These videos promoted offensive products and false messages, in some cases targeting nationalist Chinese men to boost China-Russia ties, causing emotional distress and damaging the victims' reputations.
Leonardo AI's Platform Alleged to Have Been Used for Creating Nonconsensual Celebrity Deepfakes
Sydney-based startup Leonardo AI's text-to-image generator was alleged to have been exploited to create nonconsensual sexual images of celebrities, bypassing content moderation systems with user-shared prompts.
Holmen, Wisconsin Man Allegedly Used Stable Diffusion to Create and Then Share Sexually Explicit Images Depicting Minors
The FBI has arrested Steven Anderegg of Holmen, Wisconsin for having allegedly used Stable Diffusion to generate about 13,000 sexually explicit images of minors, which he then is also alleged to have shared and distributed, including with at least one minor, via Telegram and Instagram. Anderegg was originally apprehended by state police in March, and this case marks one of the first times the FBI has brought charges against someone for having used AI to generate CSAM.
ISIS Utilizes AI for Propaganda Videos in News Harvest Program
ISIS supporters have created an AI-generated media program called News Harvest to disseminate propaganda videos. The program produces near-weekly broadcasts featuring AI-generated news anchors discussing ISIS operations globally, using cheap and easy-to-use AI tools. This development showcases the use of AI as a powerful propaganda tool for extremist groups.
Purported Deepfake Disinformation Aired on Russian State TV Allegedly Linking Ukraine to Moscow Attack
Russian state television reportedly aired a fabricated video that appeared to imitate senior Ukrainian security officials and present a false admission of involvement in the Crocus City Hall massacre. The clip reportedly used elements from recent interviews combined with allegedly AI-generated audio to create a misleading confession. ISIS-K is reported to have publicly claimed responsibility, while independent analysts and governments have rejected the narrative promoted through the video.
Investigation Reports Unauthorized Deepfake Pornography Harms Thousands of Celebrities
A Channel 4 News investigation alleges that nearly 4,000 celebrities globally, including 255 British figures, were victims of deepfake pornography. Faces were superimposed onto explicit content using AI, with the top deepfake sites garnering 100 million views in three months, according to their findings.
Manipulated Media via AI Disinformation and Deepfakes in 2024 Elections Erode Trust Across More Than 50 Countries
AI-driven election disinformation is escalating globally, leveraging easy-to-use generative AI tools to create convincing deepfakes that mislead voters. This shift has simplified the process for individuals to generate fake content, having already eroded trust in elections by undermining public trust and manipulating voter perceptions. Evidence has, for example, been documented in incidents across the U.S., Moldova, Slovakia, Bangladesh, and Taiwan.
Cartels Reportedly Using AI to Expand Operations into Financial Fraud and Human Trafficking
The Jalisco New Generation Cartel is reportedly using AI to expand its financial fraud and human trafficking operations, coercing individuals into illegal activities under the guise of legitimate jobs. INTERPOL warns that this integration of AI into criminal enterprises is a growing trend among cartels across Europe, Asia, and Africa as well.
22 Students at Richmond-Burton Community High School in Illinois Targeted by Deepfake Nudes
22 students at Richmond-Burton Community High School in Illinois were targeted in the creation of deepfake nudes. One of the students, Stevie Hyder, was targeted by classmates who used deepfake technology to alter her April 2023 prom picture into nude pictures, which were then circulated on social media. Two unnamed minors were arrested in late April 2024.
AI-Generated Images of Trump with Black Voters Spread as Disinformation Before U.S. Primary Elections
In the run-up to the U.S. primary elections, supporters of Donald Trump shared AI-generated images showing him with Black voters in an attempt to sway African-American votes. These deepfakes, including Trump's distorted hand visuals, were initially created by satirical accounts but were later misappropriated for political disinformation, misleading millions on social media platforms.
Russia-Linked AI CopyCop Site Identified as Modifying and Producing at Least 19,000 Deceptive Reports
In early March 2024, a network named CopyCop began publishing modified news stories using AI, altering content to spread partisan biases and disinformation. These articles, initially from legitimate sources, were manipulated by AI models, possibly developed by OpenAI, to disseminate Russian propaganda. Over 19,000 articles were published, targeting divisive political issues and creating false narratives.
Over 400 Purportedly AI-Driven Scams Reportedly Led to $8M Loss for Australians in 2023
In 2023, Australians reportedly lost over $8 million to scams involving purported deepfake videos and fake news articles that falsely endorsed investment trading platforms. Scammers reportedly used AI-generated content featuring celebrities to mislead victims, leading to significant financial losses. The National Anti-Scam Centre received over 400 reports of these incidents. One man is reported to have lost over $80,000 in cryptocurrency.
Fake AI-Generated Law Firms Sent Fake DMCA Notices to Increase SEO
In March 2024, fake law firms using AI-generated identities sent fraudulent DMCA takedown notices to website owners, demanding backlinks for SEO gains. These AI-generated law firms, like "Commonwealth Legal," used GAN models for realistic attorney images and fabricated bios. The scam involved fake legal threats to coerce site owners into adding backlinks, exploiting AI technology for deceptive practices.
Deepfake ID Sales Allegedly Persist as OnlyFake Reportedly Relaunches with New Fraud Tools
Researchers from Au10tix discovered the relaunch of OnlyFake, a site offering AI-generated fake IDs. Despite an earlier takedown, the site reemerged with disclaimers and new tools, including handwritten signature generation. These fakes are challenging biometric verification systems and are reportedly being used to perpetuate fraudulent activity.