AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
464 incidents
Sports Illustrated Is Alleged to Have Used AI to Invent Fake Authors and Their Articles
Sports Illustrated, managed by The Arena Group, allegedly used AI-generated authors and content, compromising journalistic integrity. Profiles of these fictitious authors, complete with AI-generated headshots, appeared alongside articles, misleading readers. The issue was exposed when inconsistencies in author identities and writing quality were noticed, leading to the removal of this content from the publication's website.
Male student allegedly used AI to generate nude photos of female classmates at a high school in Issaquah, Washington
At a high school in Issaquah, Washington, a male student is reported to have used deepfake technology to alter pictures of several female classmates and then shared them.
Reported Use of AI Voice and Identity Manipulation in the Ongoing 'Phantom Hacker' Fraud Scheme
Reports allege that updated variants of the long-running "Phantom Hacker" scam use purported AI tools to enhance impersonation, including voice cloning, spoofed caller ID, and realistic digital artifacts. Fraudsters reportedly pose as tech support, bank staff, and government officials in a three-phase scheme that pressures mostly older adults to transfer funds to accounts controlled by scammers.
Female Students at Westfield High School in New Jersey Reportedly Targeted with Deepfake Nudes
In October 2023, alleged deepfake nude images of female students were created and shared at Westfield High School in New Jersey. One alleged implicated system in this incident is the Clothoff deepfake app. A victim, Francesca Mani, and her mother, Dorota, began advocating for stronger laws after the school’s response reportedly left perpetrators unaccountable. Since the incident, their advocacy has drawn national attention, leading to trips to Washington, D.C., and bipartisan backing for legislation targeting AI-generated nonconsensual content while calling attention to deficiencies in existing legal protections.
Deepfaked Advertisements Using the Likenesses of Celebrities Such as Tom Hanks and Gayle King Without Their Consent
Deepfake technology was used to generate video advertisements featuring celebrities. Notable examples include the likeness of Tom Hanks touting a dental plan and another one in which the likeness of Gayle King touts a weight loss product. In each case, the individuals whose likenesses and voices had been deepfaked had not consented to their images and voices being used for the commercials.
Purported Deepfake Technology Was Reportedly Used to Generate Naked Pictures of Underage Girls in Spanish Town
In Spain, an AI app was used to digitally alter photos of young girls, making them appear naked. This manipulation sparked an investigation after these images were circulated in Almendralejo, a town in the Extremadura region, raising serious concerns about digital privacy violations and the potential spread of these images on pornographic sites.
Deepfake Video of Ron DeSantis Dropping Out of 2024 Presidential Race Circulating
In early September 2023, a deepfake video created by C3PMeme circulated on social media, showing Ron DeSantis falsely claiming he was dropping out of the 2024 presidential race. DeSantis did not actually suspend his campaign until January 21, 2024.
Deepfake Voice Exploit Compromises Retool's Cloud Services
In August 2023, a hacker reportedly was successful in breaching Retool, an IT company specializing in business software solutions, impacting 27 cloud customers. The attacker appears to have initiated the breach by sending phishing SMS messages to employees and later used an AI-generated deepfake voice in a phone call to obtain multi-factor authentication codes. The breach seems to have exposed vulnerabilities in Google's Authenticator app, specifically its cloud-syncing function, further enabling unauthorized access to internal systems.
Campaign Featuring Purported AI-Generated Audio Attributed to Omar al-Bashir Spreads During Sudanese Conflict
Beginning in late August 2023, a TikTok channel called The Voice of Sudan circulated purported AI-generated audio claiming to be leaked recordings of former Sudanese president Omar al-Bashir. Experts identified several recordings as voice-cloned fabrications, derived from unrelated broadcasts by a political commentator. The campaign emerged during Sudan's civil war and appeared designed to influence political perceptions. TikTok later removed the account for violating synthetic media policies.
AI-Generated Imagery and Multilingual Disinformation in Chinese Campaign Regarding Maui Wildfires
In a disinformation campaign concerning wildfires across Maui, Chinese operatives utilized AI-generated imagery to enhance the credibility of false narratives. These narratives claimed that the wildfires were the result of a secret "weather weapon" being tested by the United States. Researchers from Microsoft and other organizations identified these AI-generated images as a significant new tactic in influence operations.
Never Back Down Super PAC for Ron DeSantis Uses AI Donald Trump Voice in Attack Ad Against Kim Reynolds
A pro-Ron DeSantis super PAC released an ad featuring an AI-generated voice of Donald Trump. The ad, created by Never Back Down, aimed to criticize Trump’s treatment of Iowa Gov. Kim Reynolds. The AI-generated voice was confirmed to be based on Trump's post from his social media. This incident is an example of potential deception in political advertising through AI-generated content.
AI-Generated Articles at G/O Media Allegedly Diminishes Reputation of Human Staff
G/O Media began publishing AI-generated articles, against staff advice, that contained errors and quality issues. The first such article, a list of Star Wars movies, failed to maintain chronological order, causing internal concerns over journalistic credibility and ethics. Staff expressed that the AI was "actively hurting our reputations and credibility" and accused management of "wasting everyone's time."
Grant Reviewers Reportedly Fed Applications into Generative AI to Produce Reports, Allegedly Breaching Confidentiality
Peer reviewers of Australian government grant applications reportedly inserted applicants' work into generative AI systems such as ChatGPT to generate assessment reports, which allegedly posed confidentiality and security issues.
Amazon Rife with Many Allegedly AI-Generated Books of Suspect Quality
Amazon’s Kindle Unlimited young adult romance bestseller list was flooded with allegedly AI-generated books that made little to no sense, disrupting the rankings. These books were reported to be "clearly there to click farm." Despite being removed from the bestseller list, many remained available for purchase. The incident raised concerns about the integrity of the platform, and the potential financial impact on legitimate authors.
Alleged Exploitation of Meta's Open-Source LLaMA Model for NSFW and Violent Content
Meta's open-source large language model, LLaMA, is allegedly being used to create graphic and explicit chatbots that indulge in violent and illegal sexual fantasies. The Washington Post highlighted the example of "Allie," a chatbot that participates in text-based role-playing allegedly involving violent scenarios like rape and abuse. The issue raises ethical questions about open-source AI models, their regulation, and the responsibility of developers and deployers in mitigating harmful usage.