AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
526 incidents
FBI Reports AI Use by Threat Actors in Broader Cyber Context Including Infrastructure Intrusions
FBI Deputy Assistant Director Cynthia Kaiser stated that adversarial actors, particularly those affiliated with China and organized cybercriminal groups, are increasingly integrating AI tools across the cyberattack lifecycle, with documented use cases reportedly including purported AI-generated spear phishing, business identity fabrication, internal network mapping, and deepfake-enabled fraud. The tools are allegedly already assisting intrusions targeting U.S. infrastructure.
Meta AI App Reportedly Publishes Personal Chats Without Users Fully Realizing
Meta launched a stand-alone AI app with a "Discover" feed allowing users to share conversations with its chatbot. Multiple reports indicate that some users may have inadvertently published highly personal interactions, including audio recordings, medical questions, legal concerns, and intimate relationship disclosures. While Meta states that sharing is opt-in, the feature's design and labeling may have led to user confusion about what would be publicly visible.
Reddit Moderators Report Unauthorized AI Study Involving Fabricated Identities by Purported University of Zurich Researchers
Researchers purportedly affiliated with the University of Zurich reportedly deployed undisclosed AI-generated comments on Reddit's r/ChangeMyView to study persuasion by allegedly fabricating identities such as sexual assault survivors and racial minorities. The experiment reportedly involved unauthorized demographic profiling, emotional manipulation, and violations of subreddit and platform rules. The researchers allegedly deviated from their approved protocol without renewed ethics oversight.
Factum in Ko v. Li Allegedly Contains AI-Generated Case Law Citations
In Ko v. Li, 2025 ONSC 2766, the Ontario Superior Court of Justice identified multiple inaccurate or nonexistent case citations in a factum submitted by counsel for the applicant. Justice FL Myers questioned whether generative AI was used in drafting the document, noting the cited cases were either irrelevant, misrepresented, or did not exist. The court ordered the lawyer to show cause why she should not be cited for contempt, citing duties of accuracy and technological competence.
Anthropic Report Details Claude Misuse for Influence Operations, Credential Stuffing, Recruitment Fraud, and Malware Development
In April 2025, Anthropic published a report detailing several misuse cases involving its Claude LLM, all detected in March. These included an "influence-as-a-service" operation that orchestrated over 100 social media bots; an effort to scrape and test leaked credentials for security camera access; a recruitment fraud campaign targeting Eastern Europe; and a novice actor developing sophisticated malware. Anthropic banned the accounts involved but could not confirm downstream deployment.
Purported AI-Generated Deepfake of Steven Bartlett Reportedly Used to Promote Fake WhatsApp Investment Group
A purported deepfake video of entrepreneur and BBC Dragon's Den investor Steven Bartlett circulated on Instagram in April 2025, allegedly claiming to offer exclusive stock tips and inviting users to join a WhatsApp trading group. The reported video referenced U.S. tariff policies to entice viewers with promises of rapid returns. Bartlett's team confirmed the video was fake and worked with platforms to remove it, warning that scammers were attempting to harvest contact details for further fraud.
Scammers Reportedly Use AI Tools to Impersonate Students and Obtain Federal Aid
Between 2021 and 2025, California community colleges faced a surge in fraudulent applications—now estimated at 34% of all submissions. Reports indicate that scammers used generative AI tools, including ChatGPT, to produce identity-verifying responses that enabled them to impersonate students and obtain financial aid. The fraud allegedly resulted in over $13 million in losses over the past year alone, straining administrative and instructional systems and displacing legitimate applicants.
Purported Deepfake Video Circulated Among Students Targets Orrington, Maine Educator
An Orrington, Maine public school teacher was the target of a purported AI‑generated deepfake video depicting their likeness, according to the AOS 47 superintendent. The manipulated content was reportedly shared and circulated by some students. The district confirmed the video had no basis in reality, reported the hosting account, and had the post removed, while affirming support for the teacher and warning about the harms of AI‑based impersonation.
Alleged Deepfake Identity Scam Uses Miami Beach Realtor's Likeness to Defraud Victim in the United Kingdom in Purported Romance Scam
A scammer reportedly used deepfake video technology to impersonate Miami Beach realtor Andres Asion, creating fake videos with his face and a voice clone to catfish a woman in the UK. The victim communicated for about a year before traveling to Miami, where Asion discovered the fraud. The videos appeared convincing even to friends and family.
Microsoft Reportedly Blocks 1.6 Million Bot Signup Attempts Per Hour Amid Global AI-Driven Fraud Surge
Between April 2024 and April 2025, Microsoft reportedly blocked 1.6 million bot signups per hour and disrupted $4 billion in fraud attempts linked to AI-enhanced scams. The company's Cyber Signals report details how generative AI is being used to fabricate realistic e-commerce sites, job offers, customer service bots, and phishing lures. Fraud actors now automate mass-deceptive campaigns with fake reviews, deepfakes, and cloned brand domains at unprecedented scale and speed.
AI-Powered Presentation Tool Gamma Implicated in Multi-Stage Phishing Campaign
Attackers reportedly exploited Gamma, an AI-powered presentation tool, to create convincing presentation pages that hosted links to a spoofed Microsoft SharePoint login portal. The phishing flow allegedly used compromised email accounts, Cloudflare Turnstile for bot evasion, and adversary-in-the-middle (AiTM) tactics to validate credentials in real time and capture session cookies. The campaign aimed to bypass MFA and compromise accounts.
$31,000 Sanction in Lacey v. State Farm Tied to Purportedly Undisclosed Use of LLMs and Erroneous Citations
In the case of Lacey v. State Farm, two law firms were sanctioned $31,000 after submitting a legal brief containing reportedly erroneous citations generated using AI tools. The court reportedly found that the lawyers failed to disclose the use of AI, neglected to verify its output, and refiled a revised brief with additional inaccuracies. Judge Michael Wilner deemed the conduct reckless and issued sanctions for what he described as "improper" and "misleading" legal filings.
Alleged Deepfake Investment Scam Uses Economist David Rosenberg's Likeness on Meta Platforms
Fraudsters allegedly used AI-generated content and social media ads to impersonate economist David Rosenberg and promote a fraudulent investment scheme. Victims were reportedly directed to WhatsApp groups with fake stock tips, resulting in substantial financial losses, with some exceeding $500,000. The ads reportedly appeared on Meta platforms, including Facebook and Instagram. Rosenberg and his family reported the scam to authorities.
Reportedly Unsafe Deployment of Llama.cpp Reveals Interactive AI-Generated CSAM Roleplay Prompts
A study by UpGuard reports that misconfigured llama.cpp servers publicly exposed user prompts, including hundreds of interactive roleplay scenarios. Some prompts explicitly described fictional sexual abuse of children aged 7–12. While no real children were involved, the findings demonstrate how open-source LLMs can be exploited to generate AI-enabled child sexual abuse material (CSAM).
Alleged AI-Generated Clone of Exante Brokerage Used to Defraud U.S. Investor via JPMorgan Account
Scammers used AI tools to clone the broker Exante and defraud at least one U.S. victim by registering a JPMorgan Chase account and replicating Exante’s trading interface. AI-generated fake documents, deepfakes, and cloned websites enabled the scheme. Exante, which does not operate in the U.S., confirmed the fraud and filed reports with multiple U.S. agencies.