AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
26 incidents
Figma Disables AI Feature After Accusations of Copying Apple’s Weather App
Figma has temporarily disabled its AI design feature, "Make Design," after accusations of copying Apple’s Weather app. Andy Allen of NotBoring Software highlighted the issue, prompting Figma CEO Dylan Field to deny claims of training the AI on specific app designs. However, Field acknowledged flaws in the QA process and promised to suspend the feature until it meets quality standards. The incident has implications for designers.
Students at Australian Schools Reportedly Used AI Nudification Tools to Create Nonconsensual Sexual Images of Classmates and Teachers
Throughout 2024, schools in Australia reportedly dealt with a significant rise and proliferation of nonconsensual deepfake pornography of students. Often, male students are reported to use "nudify" apps such as Undress AI with images of their classmates and teachers. Many of the sites have reportedly remained legal and allegedly accessible to minors, who in turn are purportedly using the sites to generate pornography of their peers.
TikTok Hosts AI-Generated English-Language Hitler Speeches with Millions of Views
AI-generated English-language Adolf Hitler speeches have been proliferating on TikTok. They are reportedly accumulating millions of views despite violating the platform’s hate speech policies. The clips are described as often pairing the audio with misleading translations and memes that glorify Hitler and distort historical facts. While some content has been removed, many accounts reportedly continue to post similar videos.
Scams Reportedly Impersonating Wealthy Investors Proliferating on Facebook
Scams are reportedly proliferating throughout Facebook impersonating wealthy individuals such as Bill Ackman, Cathie Wood, Steve Cohen, Peter Lynch, and Ray Dalio. In some cases, it seems deepfake technology is being employed, while simultaneously Facebook's own AI systems are allegedly faltering in their ability to halt the spread of these fraudulent ads despite being reported.
Fraudsters Use Deepfake Video of Tim Cook to Promote Apple Crypto Scam on YouTube
Fraudsters repurposed an old interview with Apple CEO Tim Cook using AI or video editing to promote a fake crypto event on YouTube. The altered video was designed to mislead viewers into believing Tim Cook endorsed a new cryptocurrency scheme. The stream attracted tens of thousands of viewers, potentially exposing them to the scam before it was removed for violating YouTube’s terms of service.
Facebook Recommended Military Gear Ads Despite Pause on Weapons Accessories Ads
Facebook’s algorithmic recommendations reportedly continued showing advertisements for gun accessories and military gear, despite Facebook’s halt on weapons accessories ads following the US Capitol attack.
Noelle Martin Deepfaked Without Consent in AI-Generated Pornography
In 2017, Noelle Martin discovered explicit deepfake videos online that used AI technology to superimpose her face onto pornographic scenes. This incident was a continuation of the abuse she had experienced since at least 2012, when she first found doctored still images of herself in similar contexts. Despite the initial lack of legal protections, her advocacy efforts were instrumental in making image-based abuse a criminal offense in Australia.
Self-Employed Somerset Father Wayne Luxon Reportedly Lost £140,000 After Facebook Deepfake of Martin Lewis Promoted Crypto Scam
Beginning in 2020, Wayne Luxon, a 43-year-old self-employed father of four from Taunton, Somerset, reportedly lost £140,000 after responding to a Facebook advertisement using a purportedly AI-generated Martin Lewis deepfake to promote cryptocurrency investment. Luxon said he was directed to a clone platform where he made additional payments after seeing fictitious gains, and later paid purported tax fees before the account balance fell to zero.
Amazon Reportedly Recommends Explosive-Producing Ingredients as 'Frequently Bought Together' Items for Chemicals
Amazon was reported to have shown chemical combinations for producing explosives and incendiary devices as "frequently bought together" items via automated recommendation.
YouTube Recommendations Allegedly Promoted Radicalizing Material Contributing to Terrorist Acts
Family of Nohemi Gonzalez alleged YouTube recommendation systems led people to propaganda videos for the Islamic State which subsequently radicalized them to carry out the killing of 130 people in the 2015 Paris terrorist attack, including Ms. Gonzalez.
YouTube Recommendations Implicated in Political Radicalization of User
YouTube’s personalization and recommendation algorithms were alleged to have pushed and exposed its young male users to political extremism and misinformation, driving them towards far-right ideologies such as neo-Nazism and white supremacy.