AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
197 incidents
Kenyan Journalist Jeff Koinange Depicted Endorsing Gambling App in Purported AI-Generated Deepfake
A viral video purportedly manipulated by AI shows Kenyan journalist Jeff Koinange endorsing a mobile gambling app. The video was reportedly created using AI deepfake techniques, uses footage from a legitimate news broadcast, and artificially alters Koinange's voice and mouth movements. Neither Koinange nor Citizen TV has promoted the app, and no legitimate source hosts the video.
Alleged Deepfake Audio Depicts Philippines President Ferdinand Marcos Jr. Ordering Military Action
A purported deepfake audio clip portraying Philippine President Ferdinand Marcos Jr. ordering an attack on China spread online in July 2024, fueling tensions in the West Philippine Sea. The Presidential Communications Office reportedly debunked it as fake, attributing it to foreign actors.
Child Predators Are Reportedly Generating Deepfake Nudes of Children to Extort Them
Internet Watch Foundation (IWF) has reported that it has found a manual on the dark web that encourages criminals to use "nudifying" AI tools to depict children naked in order to extort victims into providing graphic content.
Amandine Le Pen Deepfake Account Misleads Thousands on TikTok
A TikTok account, "Amandine Le Pen," created using AI deepfake technology, impersonated a fictional niece of Marine Le Pen, amassing over 30,000 followers. The account spread pro-RN messages and solicited donations, misleading users and exploiting political influence. Visual inconsistencies revealed the deepfake, raising concerns about AI misuse for political manipulation, identity theft, and violation of personal rights, especially with similar accounts proliferating, such as "Lena Maréchal Lepen."
Reported Audio Deepfake Impersonating CEO Karim Toubba Targets LastPass Employee via WhatsApp
A LastPass employee was reportedly targeted by an audio deepfake impersonating CEO Karim Toubba via WhatsApp. The message used voice-cloning AI in a social engineering attempt to create urgency and bypass security protocols. The employee recognized red flags, reported the incident, and no breach occurred. LastPass shared the incident publicly to raise awareness about deepfake-driven fraud campaigns targeting executives and companies.
Deepfake Porn Sites Use Breeze Liu's Image Without Consent
Porn sites are alleged to have used AI-generated images of Breeze Liu without her consent, leading to severe emotional distress. Liu discovered a video of herself on Pornhub, which was then deepfaked and spread across over 800 links. Despite efforts to remove the content, many sites refused to comply, perpetuating the violation and exploitation of her image.
TikTok Hosts AI-Generated English-Language Hitler Speeches with Millions of Views
AI-generated English-language Adolf Hitler speeches have been proliferating on TikTok. They are reportedly accumulating millions of views despite violating the platform’s hate speech policies. The clips are described as often pairing the audio with misleading translations and memes that glorify Hitler and distort historical facts. While some content has been removed, many accounts reportedly continue to post similar videos.
Reportedly Viral AI-Generated Song about 'Diddy Party' Allegedly Mimics Justin Bieber
A purportedly AI-generated song imitating Justin Bieber's voice, allegedly referencing a "Diddy party," spread widely across social media, reportedly leading fans to believe it was authentic. Experts reportedly identified the audio as likely AI-made, citing frequency mismatches and digital artifacts. The reportedly viral song misrepresented Bieber, posing potential brand and revenue impacts for the artist while misleading listeners.
China Reportedly Intensifying AI to Spread Disinformation to U.S. and Taiwanese Voters
AI tools linked to China were used to disseminate disinformation targeting voters in the U.S. and Taiwan, according to a Microsoft report. These operations included AI-generated imagery and audio to influence political perceptions and election outcomes, originating from the APT Storm-1376 (also known as Spamouflage and Dragonbridge).
'Lavender' and 'The Gospel' AI Systems Reportedly Used in Gaza Targeting Operations with Civilian Harm Allegations
The AI system known as "Lavender" was reportedly used by the Israel Defense Forces (IDF) to assist in identifying individuals in Gaza for targeting, while a related system, "The Gospel," was reportedly used to help select and prioritize physical strike targets. Investigations reportedly suggest that these systems operated with limited human review and may have contributed to strikes associated with high civilian casualties. The extent of automation with human oversight, as well as the accuracy of these targeting processes, reportedly remain subjects of dispute.
Investigation Reports Unauthorized Deepfake Pornography Harms Thousands of Celebrities
A Channel 4 News investigation alleges that nearly 4,000 celebrities globally, including 255 British figures, were victims of deepfake pornography. Faces were superimposed onto explicit content using AI, with the top deepfake sites garnering 100 million views in three months, according to their findings.
Manipulated Media via AI Disinformation and Deepfakes in 2024 Elections Erode Trust Across More Than 50 Countries
AI-driven election disinformation is escalating globally, leveraging easy-to-use generative AI tools to create convincing deepfakes that mislead voters. This shift has simplified the process for individuals to generate fake content, having already eroded trust in elections by undermining public trust and manipulating voter perceptions. Evidence has, for example, been documented in incidents across the U.S., Moldova, Slovakia, Bangladesh, and Taiwan.
Fake AI-Generated Law Firms Sent Fake DMCA Notices to Increase SEO
In March 2024, fake law firms using AI-generated identities sent fraudulent DMCA takedown notices to website owners, demanding backlinks for SEO gains. These AI-generated law firms, like "Commonwealth Legal," used GAN models for realistic attorney images and fabricated bios. The scam involved fake legal threats to coerce site owners into adding backlinks, exploiting AI technology for deceptive practices.
Deepfake ID Sales Allegedly Persist as OnlyFake Reportedly Relaunches with New Fraud Tools
Researchers from Au10tix discovered the relaunch of OnlyFake, a site offering AI-generated fake IDs. Despite an earlier takedown, the site reemerged with disclaimers and new tools, including handwritten signature generation. These fakes are challenging biometric verification systems and are reportedly being used to perpetuate fraudulent activity.
Nonconsensual Deepfake Porn of Bobbi Althoff Spreads Rapidly on X
Nonconsensual deepfake pornography of Bobbi Althoff, which had been in circulation for six months, is reported to have suddenly gone viral on X, jumping from around 178,000 views to 6.5 million views over a matter of hours. In addition to the harm to Althoff, this incident also spotlights X's role in distributing AI-generated nonconsensual porn due to alleged lax moderation.