AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
538 incidents
Non-consensual Deepfake Porn Targeted Female Content Creators
Unauthorized, non-consensual deepfake pornography showing faces of high-profile female streamers and content creators was published on a subscription-based website, which gained notoriety after a male streamer was caught accessing the site.
Purported Synthesia Avatars Used in Alleged Pro-Junta Deepfake Video Supporting Ibrahim Traoré in Burkina Faso
In January 2023, purportedly AI-generated videos reportedly began circulating on WhatsApp and social media showing avatars of apparent American pan-Africanists expressing support for Burkina Faso's military junta. The videos were reportedly created using the Synthesia platform. Synthesia later confirmed the videos violated its terms of service and banned the user. The source of the videos remains unidentified.
Mother in Arizona Received Fake Ransom Call Featuring AI Voice of Her Daughter
A mother in Arizona received a ransom call from an anonymous scammer who created her daughter's voice allegedly using AI voice synthesis, which was proven to be fake once her daughter's safety was confirmed.
ChatGPT-Written Bug Reports Deemed "Nonsense" by White Hat Platform, Prompted Bans
ChatGPT-generated responses submitted to smart contract bug bounty platform Immunefi reportedly lacked details to help diagnose technical issues, which reportedly wasted the platform's time, prompting bans to submitters.
Canadian Parents Tricked out of Thousands Using Their Son's AI Voice
Two Canadian residents were scammed by an anonymous caller who used AI voice synthesis to replicate their son's voice asking them for legal fees, disguising as his lawyer.
ChatGPT Abused to Develop Malicious Softwares
OpenAI's ChatGPT was reportedly abused by cyber criminals including ones with no or low levels of coding or development skills to develop malware, ransomware, and other malicious softwares.
AI Video-Making Tool Abused to Deploy Pro-China News on Social Media
Synthesia's AI-generated video-making tool was reportedly used by Spamouflage to disseminate pro-China propaganda news on social media using videos featuring highly realistic fictitious news anchors.
Chinese Accounts Spammed Twitter Feed Allegedly to Obscure News of Protests
Twitter Feed was flooded by content from Chinese-language accounts which allegedly aimed to manipulate and reduce social media coverage about widespread protests against coronavirus restrictions in China.
Deepfake of FTX's Former CEO Posted on Twitter Aiming to Scam FTX Collapse Victims
A visual and audio deepfake of former FTX CEO Sam Bankman-Fried was posted on Twitter to scam victims of the exchange's collapse by urging people to transfer funds into an anonymous cryptocurrency wallet.
Kate Isaacs, Advocate Against Image-Based Abuse, Reports Being Deepfaked
Kate Isaacs, a London-based activist and founder of the #NotYourPorn campaign, was targeted in a deepfake incident. Her face was alleged to have been digitally manipulated onto a pornographic video using AI and shared online. The reported video, tagged with her name, is alleged to have led to streams of abuse, doxing, and threats of violence. The attack reportedly followed her efforts to pressure PornHub to remove unverified content.
Weibo Model Had Difficulty Detecting Shifts in Censored Speech
Weibo's user moderation model is having difficulty keeping up with shifting user slang in defiance of Chinese state censors.
Students Reportedly Used Generative Text Tools to Complete School Assignments and Exams
Students in school and university settings reportedly used generative text tools including Sudowrite, GPT-3/OpenAI Playground, and ChatGPT to complete homework, essays, and exams without disclosure. Reporting described admitted student use and growing concern among educators that these tools undermined academic integrity, disadvantaged non-using students, and disrupted assessment practices.
Stable Diffusion Abused by 4chan Users to Deepfake Celebrity Porn
Stable Diffusion, an open-source image generation model by Stability AI, was reportedly leaked on 4chan prior to its release date, and was used by its users to generate pornographic deepfakes of celebrities.
Deepfakes Reportedly Deployed in Online Interviews for Remote Work Positions
Voice and video deepfakes were reported by FBI Internet Crime Complaint Center (IC3) in complaint reports to have been deployed during online interviews of the candidates for remote-work positions.
Yahoo Boys Allegedly Employ Real-Time Deepfake Technology in Romance Scams
Scammers from Nigeria, known as "Yahoo Boys," are reportedly utilizing real-time deepfake technology to impersonate individuals during video calls, deceiving victims in romance scams. By allegedly altering their appearance with face-swapping software, they build trust under false identities to defraud targets of substantial sums.