AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
368 incidents
Students at a Beverly Hills Middle School Allegedly Created and Shared Deepfake Nudes of Their Classmates
At Beverly Vista Middle School in Beverly Hills, California, students allegedly used AI to generate fake nude photos with their classmates' faces, prompting investigations by school officials and the police. The incident highlights the increasing misuse of generative AI among minors.
Two Florida Middle School Students Arrested Under New Law for Allegedly Having Made and Shared Deepfake Nudes of Their Classmates
Two teenaged boys from Miami, Florida, were arrested for allegedly creating and sharing AI-generated nude images of their classmates. Charged under a 2022 Florida law, they face third-degree felonies for producing and disseminating altered sexual depictions.
Underground Market for LLMs Powers Malware and Phishing Scams
A study by Indiana University researchers uncovered widespread misuse of large language models (LLMs) for cybercrime. Cybercriminals, according to that study, use LLMs like OpenAI's GPT-3.5 and GPT-4 to create malware, phishing scams, and scam websites. These models are available on underground markets, often bypassing safety checks through jailbreaking. Named malicious LLMs are BadGPT, XXXGPT, Evil-GPT, WormGPT, FraudGPT, BLACKHATGPT, EscapeGPT, DarkGPT, and WolfGPT.
Male student allegedly used AI to generate nude photos of female classmates at a high school in Issaquah, Washington
At a high school in Issaquah, Washington, a male student is reported to have used deepfake technology to alter pictures of several female classmates and then shared them.
Reported Use of AI Voice and Identity Manipulation in the Ongoing 'Phantom Hacker' Fraud Scheme
Reports allege that updated variants of the long-running "Phantom Hacker" scam use purported AI tools to enhance impersonation, including voice cloning, spoofed caller ID, and realistic digital artifacts. Fraudsters reportedly pose as tech support, bank staff, and government officials in a three-phase scheme that pressures mostly older adults to transfer funds to accounts controlled by scammers.
Female Students at Westfield High School in New Jersey Reportedly Targeted with Deepfake Nudes
In October 2023, alleged deepfake nude images of female students were created and shared at Westfield High School in New Jersey. One alleged implicated system in this incident is the Clothoff deepfake app. A victim, Francesca Mani, and her mother, Dorota, began advocating for stronger laws after the school’s response reportedly left perpetrators unaccountable. Since the incident, their advocacy has drawn national attention, leading to trips to Washington, D.C., and bipartisan backing for legislation targeting AI-generated nonconsensual content while calling attention to deficiencies in existing legal protections.
Deepfaked Advertisements Using the Likenesses of Celebrities Such as Tom Hanks and Gayle King Without Their Consent
Deepfake technology was used to generate video advertisements featuring celebrities. Notable examples include the likeness of Tom Hanks touting a dental plan and another one in which the likeness of Gayle King touts a weight loss product. In each case, the individuals whose likenesses and voices had been deepfaked had not consented to their images and voices being used for the commercials.
Purported Deepfake Technology Was Reportedly Used to Generate Naked Pictures of Underage Girls in Spanish Town
In Spain, an AI app was used to digitally alter photos of young girls, making them appear naked. This manipulation sparked an investigation after these images were circulated in Almendralejo, a town in the Extremadura region, raising serious concerns about digital privacy violations and the potential spread of these images on pornographic sites.
Deepfake Voice Exploit Compromises Retool's Cloud Services
In August 2023, a hacker reportedly was successful in breaching Retool, an IT company specializing in business software solutions, impacting 27 cloud customers. The attacker appears to have initiated the breach by sending phishing SMS messages to employees and later used an AI-generated deepfake voice in a phone call to obtain multi-factor authentication codes. The breach seems to have exposed vulnerabilities in Google's Authenticator app, specifically its cloud-syncing function, further enabling unauthorized access to internal systems.
Campaign Featuring Purported AI-Generated Audio Attributed to Omar al-Bashir Spreads During Sudanese Conflict
Beginning in late August 2023, a TikTok channel called The Voice of Sudan circulated purported AI-generated audio claiming to be leaked recordings of former Sudanese president Omar al-Bashir. Experts identified several recordings as voice-cloned fabrications, derived from unrelated broadcasts by a political commentator. The campaign emerged during Sudan's civil war and appeared designed to influence political perceptions. TikTok later removed the account for violating synthetic media policies.
AI-Generated Imagery and Multilingual Disinformation in Chinese Campaign Regarding Maui Wildfires
In a disinformation campaign concerning wildfires across Maui, Chinese operatives utilized AI-generated imagery to enhance the credibility of false narratives. These narratives claimed that the wildfires were the result of a secret "weather weapon" being tested by the United States. Researchers from Microsoft and other organizations identified these AI-generated images as a significant new tactic in influence operations.
Grant Reviewers Reportedly Fed Applications into Generative AI to Produce Reports, Allegedly Breaching Confidentiality
Peer reviewers of Australian government grant applications reportedly inserted applicants' work into generative AI systems such as ChatGPT to generate assessment reports, which allegedly posed confidentiality and security issues.
Amazon Rife with Many Allegedly AI-Generated Books of Suspect Quality
Amazon’s Kindle Unlimited young adult romance bestseller list was flooded with allegedly AI-generated books that made little to no sense, disrupting the rankings. These books were reported to be "clearly there to click farm." Despite being removed from the bestseller list, many remained available for purchase. The incident raised concerns about the integrity of the platform, and the potential financial impact on legitimate authors.
Ron DeSantis's Presidential Campaign Released Twitter Video Containing AI Images of Donald Trump Hugging Anthony Fauci
Ron DeSantis’s presidential campaign shared a video on Twitter featuring some AI-generated images of Donald Trump hugging former White House coronavirus advisor Anthony Fauci, allegedly as a smear campaign. This incident is possibly the first time a major U.S. presidential campaign deployed deepfakes with the intention of misleading the electorate.
AI-Generated Voices Amplify Conspiracy Theories on TikTok
NewsGuard has identified 17 TikTok accounts that have been using AI-generated voices to advance and amplify conspiracy theories and false claims beginning in June 2023. By September 25, 2023, these accounts had amassed over 336 million views and over 14.5 million likes. Videos include baseless claims involving public figures such as Barack Obama, Oprah Winfrey, and Jamie Foxx.