AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
271 incidents
Manipulated Media via AI Disinformation and Deepfakes in 2024 Elections Erode Trust Across More Than 50 Countries
AI-driven election disinformation is escalating globally, leveraging easy-to-use generative AI tools to create convincing deepfakes that mislead voters. This shift has simplified the process for individuals to generate fake content, having already eroded trust in elections by undermining public trust and manipulating voter perceptions. Evidence has, for example, been documented in incidents across the U.S., Moldova, Slovakia, Bangladesh, and Taiwan.
Fake AI-Generated Law Firms Sent Fake DMCA Notices to Increase SEO
In March 2024, fake law firms using AI-generated identities sent fraudulent DMCA takedown notices to website owners, demanding backlinks for SEO gains. These AI-generated law firms, like "Commonwealth Legal," used GAN models for realistic attorney images and fabricated bios. The scam involved fake legal threats to coerce site owners into adding backlinks, exploiting AI technology for deceptive practices.
Deepfake ID Sales Allegedly Persist as OnlyFake Reportedly Relaunches with New Fraud Tools
Researchers from Au10tix discovered the relaunch of OnlyFake, a site offering AI-generated fake IDs. Despite an earlier takedown, the site reemerged with disclaimers and new tools, including handwritten signature generation. These fakes are challenging biometric verification systems and are reportedly being used to perpetuate fraudulent activity.
Nonconsensual Deepfake Porn of Bobbi Althoff Spreads Rapidly on X
Nonconsensual deepfake pornography of Bobbi Althoff, which had been in circulation for six months, is reported to have suddenly gone viral on X, jumping from around 178,000 views to 6.5 million views over a matter of hours. In addition to the harm to Althoff, this incident also spotlights X's role in distributing AI-generated nonconsensual porn due to alleged lax moderation.
Purported AI-Cloned Voice Depicts Kenyan President William Ruto Speaking French in Viral Video
A purportedly AI-manipulated video widely shared on TikTok and Facebook presents Kenyan President William Ruto speaking in French during a speech in Tokyo on February 7, 2024. A reported reverse image search confirmed the original footage shows Ruto speaking in English to the Kenyan diaspora. The reportedly manipulated version used AI-generated French audio over the original video, making it a case of voice cloning rather than a full audiovisual deepfake.
Deepfakes Circulating and Eroding Electoral Integrity in the Lead-up to 2024 South Korean legislative election
In the lead-up to Korea's parliamentary elections, at least 129 deepfake videos and images were reported to have been detected, violating new election laws. These AI-generated deepfakes were used to mislead and manipulate public opinion, prompting a crackdown by the National Election Commission.
AI Romance Apps Reportedly Compromise User Privacy for Data Harvesting
AI-powered romantic chatbots, marketed for enhancing mental health, are found to exploit user privacy by harvesting sensitive personal information for data sharing and targeted ads, with inadequate security measures and consent protocols, according to research by the Mozilla Foundation.
Deepfake Audio Falsely Attributes Controversial Remarks to Keir Starmer About the Rochdale Azhar Ali Crisis
A deepfake audio clip, falsely claiming to be Keir Starmer discussing the Rochdale byelection and Labour's withdrawl of support for Azhar Ali, circulated online, achieving over 250,000 views. Experts confirmed its inauthenticity, highlighting a significant misuse of AI in fabricating political content.
Purported Deepfake Video Allegedly Claims Kyiv's Assassination Plan Against President Emmanuel Macron
A purported deepfake video reportedly claimed France 24 reported a Kyiv plot to assassinate French President Macron. This reported fake news was debunked by France 24, which confirmed the video was altered and did not air any such report.
Alleged Deepfake Audio of Imran Khan Calls for Election Boycott, Misleading Pakistan Voters
A purported deepfake audio clip, falsely attributed to Imran Khan urging a PTI (Pakistan Tehreek-e-Insaf) election boycott, circulated on social media on the eve of Pakistan's general elections. This sophisticated AI-generated misinformation aimed to mislead voters, highlighting the growing challenge of digital manipulation in political discourse.
Purported Russian-Linked Network Allegedly Used Deepfake of Maria Ressa on Facebook and Bing to Promote Cryptocurrency Scam Targeting Filipinos
A purportedly Russian-linked scam network allegedly circulated an AI-generated deepfake of journalist Maria Ressa on Facebook and Microsoft Bing. The video reportedly manipulated a 2022 interview to falsely depict Ressa endorsing cryptocurrency. Fraudulent websites allegedly impersonating Rappler and CNN Philippines reportedly amplified the scam. Investigators reportedly linked the operation to a fraudulent network targeting Filipino audiences. Meta and Microsoft removed the content, but similar scams allegedly continue to resurface.
Alleged Deepfake CFO Scam Reportedly Costs Multinational Engineering Firm Arup $25 Million
A finance employee at the multinational engineering firm Arup was reportedly deceived into transferring $25 million by fraudsters using purported deepfake technology to impersonate the firm's CFO in a video call, according to the Hong Kong police.
AI-Generated Fake News Targets Black Celebrities on YouTube
YouTube faced a surge of AI-generated fake news targeting Black celebrities, including fake narratives about Sean “Diddy” Combs and others. These videos, blending AI-generated and manipulated media, amassed millions of views, challenging content moderation efforts and highlighting the spread of disinformation.
Purported Deepfake of Andrew Forrest Used to Promote Fraudulent 'Quantum AI' Crypto Platform on Facebook
Cybertrace reportedly identified a purported deepfake of Australian billionaire Andrew Forrest circulating on Facebook on January 27, 2024, falsely depicting him endorsing a fraudulent crypto platform called Quantum AI. The video reportedly altered footage from a 2023 Rhodes Trust talk. Forrest reportedly condemned Meta for failing to curb AI-generated frauds.
Significant Increase in Deepfake Nudes of Taylor Swift Circulating on Social Media
AI-generated sexually explicit images of Taylor Swift circulated on X, garnering over 45 million views before removal. Originating from a Telegram group, these deepfakes challenge content moderation, as X's policies against synthetic media and nonconsensual nudity were violated.