AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
526 incidents
Proliferation of Deepfakes Disrupting 2024 Lok Sabha Elections
Digital manipulators in India are using deepfake technology to influence the 2024 Lok Sabha elections. These AI-generated videos and audio clips are designed to tarnish the reputations of political candidates, challenging the integrity of electoral processes.
Social Media Scammers Used Deepfakes of Taylor Swift and Several Other Celebrities in Fraudulent Le Creuset Cookware Giveaways
Scammers reportedly made deepfakes of Taylor Swift, Selena Gomez, Joanna Gaines, Lainey Wilson, Ree Drummond, Oprah, Jennifer Lopez, Trisha Yearwood, Martha Stewart, and Blake Shelton promoting a Le Creuset giveaway. These AI-generated ads, appearing on Meta and TikTok, falsely claimed users could receive free cookware by paying a small shipping fee. Victims were unknowingly enrolled in a costly monthly subscription.
Child Sexual Abuse Material Taints Image Generators
The LAION-5B dataset (a commonly used dataset with more than 5 billion image-description pairs) was found by researchers to contain child sexual abuse material (CSAM), which increases the likelihood that downstream models will produce CSAM imagery. The discovery taints models built with the LAION dataset requiring many organizations to retrain those models. Additionally, LAION must now scrub the dataset of the imagery.
Deepfake Nudes Targeting Underage Female Students at Collège Béliveau in Winnipeg Shared Online
At Collège Béliveau in Winnipeg, female students between grades 7-12 were targeted in the creation of deepfake nudes, which were then distributed online. Specific numbers and identities of victims and perpetrators were not released, and no charges were ultimately filed owing to the gap between existing laws and the nature of the incident.
Students at a Beverly Hills Middle School Allegedly Created and Shared Deepfake Nudes of Their Classmates
At Beverly Vista Middle School in Beverly Hills, California, students allegedly used AI to generate fake nude photos with their classmates' faces, prompting investigations by school officials and the police. The incident highlights the increasing misuse of generative AI among minors.
Two Florida Middle School Students Arrested Under New Law for Allegedly Having Made and Shared Deepfake Nudes of Their Classmates
Two teenaged boys from Miami, Florida, were arrested for allegedly creating and sharing AI-generated nude images of their classmates. Charged under a 2022 Florida law, they face third-degree felonies for producing and disseminating altered sexual depictions.
OpenDream AI Platform Reportedly Commercialized AI-Generated CSAM and Non-consensual Deepfake Sexual Images
OpenDream, an AI image generation platform operated by CBM Media Pte Ltd, reportedly allowed users to generate and monetize AI-generated CSAM and non-consensual sexual deepfakes through its public gallery and paid NSFW tools. The content was reported to be publicly accessible and indexed by search engines for months before limited moderation occurred in mid-2024, prompting takedowns by Google and Bing and the termination of its ad and payment services.
Underground Market for LLMs Powers Malware and Phishing Scams
A study by Indiana University researchers uncovered widespread misuse of large language models (LLMs) for cybercrime. Cybercriminals, according to that study, use LLMs like OpenAI's GPT-3.5 and GPT-4 to create malware, phishing scams, and scam websites. These models are available on underground markets, often bypassing safety checks through jailbreaking. Named malicious LLMs are BadGPT, XXXGPT, Evil-GPT, WormGPT, FraudGPT, BLACKHATGPT, EscapeGPT, DarkGPT, and WolfGPT.
Sports Illustrated Is Alleged to Have Used AI to Invent Fake Authors and Their Articles
Sports Illustrated, managed by The Arena Group, allegedly used AI-generated authors and content, compromising journalistic integrity. Profiles of these fictitious authors, complete with AI-generated headshots, appeared alongside articles, misleading readers. The issue was exposed when inconsistencies in author identities and writing quality were noticed, leading to the removal of this content from the publication's website.
Male student allegedly used AI to generate nude photos of female classmates at a high school in Issaquah, Washington
At a high school in Issaquah, Washington, a male student is reported to have used deepfake technology to alter pictures of several female classmates and then shared them.
Reported Use of AI Voice and Identity Manipulation in the Ongoing 'Phantom Hacker' Fraud Scheme
Reports allege that updated variants of the long-running "Phantom Hacker" scam use purported AI tools to enhance impersonation, including voice cloning, spoofed caller ID, and realistic digital artifacts. Fraudsters reportedly pose as tech support, bank staff, and government officials in a three-phase scheme that pressures mostly older adults to transfer funds to accounts controlled by scammers.
Female Students at Westfield High School in New Jersey Reportedly Targeted with Deepfake Nudes
In October 2023, alleged deepfake nude images of female students were created and shared at Westfield High School in New Jersey. One alleged implicated system in this incident is the Clothoff deepfake app. A victim, Francesca Mani, and her mother, Dorota, began advocating for stronger laws after the school’s response reportedly left perpetrators unaccountable. Since the incident, their advocacy has drawn national attention, leading to trips to Washington, D.C., and bipartisan backing for legislation targeting AI-generated nonconsensual content while calling attention to deficiencies in existing legal protections.
Deepfaked Advertisements Using the Likenesses of Celebrities Such as Tom Hanks and Gayle King Without Their Consent
Deepfake technology was used to generate video advertisements featuring celebrities. Notable examples include the likeness of Tom Hanks touting a dental plan and another one in which the likeness of Gayle King touts a weight loss product. In each case, the individuals whose likenesses and voices had been deepfaked had not consented to their images and voices being used for the commercials.
Purported Deepfake Technology Was Reportedly Used to Generate Naked Pictures of Underage Girls in Spanish Town
In Spain, an AI app was used to digitally alter photos of young girls, making them appear naked. This manipulation sparked an investigation after these images were circulated in Almendralejo, a town in the Extremadura region, raising serious concerns about digital privacy violations and the potential spread of these images on pornographic sites.
Deepfake Video of Ron DeSantis Dropping Out of 2024 Presidential Race Circulating
In early September 2023, a deepfake video created by C3PMeme circulated on social media, showing Ron DeSantis falsely claiming he was dropping out of the 2024 presidential race. DeSantis did not actually suspend his campaign until January 21, 2024.