AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
49 incidents
COEMPT Quality Assurance Engineers Allegedly Violated Indian CBSE Student Data Privacy Rights by Processing It with Google Gemini
The Hindu reported that vulnerabilities in the OnMark exam-marking portal used by India's Central Board of Secondary Education (CBSE) allegedly exposed sensitive student data, including answer-sheet images. Ethical hacker Nisarga Adhikary also alleged that COEMPT Eduteck quality-assurance scripts processed students' personal information through Google Gemini. CBSE said the vulnerabilities had been contained.
Hidden Prompt Injection in Brazilian Labor-Court Petition Reportedly Tried to Manipulate Galileu
Galileu, an AI tool used by Brazil's labor courts, reportedly detected hidden instructions embedded in an initial petition before the 3rd Labor Court of Parauapebas. The text allegedly told the AI to contest the petition superficially and not challenge documents. Galileu reportedly alerted the judge and blocked the hidden content from processing; the judge then reviewed the material before imposing any procedural consequences.
Anthropic Said DeepSeek, Moonshot, and MiniMax Used Fraudulent Accounts and Proxies to Illicitly Distill Claude Capabilities at Scale
Anthropic said it identified large-scale campaigns that used fraudulent accounts and proxy services to generate high volumes of Claude interactions to extract model capabilities for competitor training ("distillation"). Anthropic attributed the activity to DeepSeek, Moonshot, and MiniMax and said it involved millions of exchanges across thousands of accounts, violating its terms and access restrictions. Anthropic described detection measures, account controls, and indicator-sharing in response.
Moltbook Database Exposure Allegedly Revealed Users' Private Communications and API Authentication Tokens
Wiz researchers reported accessing an exposed Moltbook database in under three minutes, allegedly obtaining ~35,000 email addresses, thousands of private DMs, and ~1.5 million API authentication tokens. The exposure was described as enabling read/write access and potential impersonation or manipulation of "AI agent" accounts. Wiz said it disclosed the issue to Moltbook, which reportedly secured the database within hours and deleted accessed data.
Secret Desires AI Platform Reportedly Exposed Nearly Two Million Sensitive Images in Cloud Storage Leak
The erotic AI chatbot and image-generation platform Secret Desires reportedly left nearly two million sensitive images and videos publicly exposed in misconfigured cloud storage. The leaked files reportedly included personal photos, workplace and university information, and explicit AI-generated deepfakes of women and girls. The content reportedly became inaccessible shortly after journalists contacted the platform.
Alleged Malicious Wiping Command Found in Amazon Q AI Assistant
A reported compromise of Amazon's AI coding assistant "Q" allegedly involved the insertion of commands that, if executed, could have wiped local files and potentially affected cloud resources. The altered code was reportedly incorporated into a public release before being detected and removed.
CISA Acting Director Reportedly Uploaded Sensitive Government Documents to Public ChatGPT Instance
Madhu Gottumukkala, acting director of the Cybersecurity and Infrastructure Security Agency (CISA), reportedly uploaded government contracting documents marked "for official use only" into a public version of ChatGPT. The uploads reportedly triggered automated cybersecurity alerts and prompted a Department of Homeland Security review to assess potential exposure of sensitive information.
Urban VPN Proxy Browser Extension Reportedly Harvested and Sold Private AI Chatbot Conversations via Silent Update
Security researchers reported that the Urban VPN Proxy browser extension introduced AI conversation–harvesting functionality in version 5.5.0, released July 9, 2025. The extension allegedly intercepted and exfiltrated private conversations from AI platforms including ChatGPT, Claude, Gemini, Grok, and others without a user-facing opt-out. The data, including sensitive personal and financial information, was reportedly shared with affiliated data brokers for commercial analytics.
McDonald's McHire AI Recruitment Platform Reportedly Exposed Data of 64 Million Applicants via Default Login and API Vulnerability
Researchers Ian Carroll and Sam Curry reported that McDonald's AI-powered hiring tool, McHire (using Paradox.ai's "Olivia" chatbot), could purportedly be accessed via default admin credentials and an insecure direct object reference in an internal API. The flaws allegedly allowed viewing of applicants' personally identifiable information and chat histories. McDonald's and Paradox reportedly patched the issues within a day of disclosure; Paradox stated only five records were accessed.
Voice Actor Alleges Unconsented Use of AI-Generated Voice on ScotRail Trains
Scottish voice actor Gayanne Potter alleges her voice was used without proper consent in ScotRail's AI train announcements. She claims she had agreed to limited use of her voice data by ReadSpeaker but was not informed it would be used in a synthetic voice system called "Iona." ScotRail continues to use the voice, stating the dispute is between Potter and ReadSpeaker.
Meta AI App Reportedly Publishes Personal Chats Without Users Fully Realizing
Meta launched a stand-alone AI app with a "Discover" feed allowing users to share conversations with its chatbot. Multiple reports indicate that some users may have inadvertently published highly personal interactions, including audio recordings, medical questions, legal concerns, and intimate relationship disclosures. While Meta states that sharing is opt-in, the feature's design and labeling may have led to user confusion about what would be publicly visible.
Reportedly Unsafe Deployment of Llama.cpp Reveals Interactive AI-Generated CSAM Roleplay Prompts
A study by UpGuard reports that misconfigured llama.cpp servers publicly exposed user prompts, including hundreds of interactive roleplay scenarios. Some prompts explicitly described fictional sexual abuse of children aged 7–12. While no real children were involved, the findings demonstrate how open-source LLMs can be exploited to generate AI-enabled child sexual abuse material (CSAM).
GenNomis AI Database Reportedly Exposes Nearly 100,000 Deepfake and Nudify Images in Public Breach
In March 2025, cybersecurity researcher Jeremiah Fowler discovered an unprotected database linked to GenNomis by AI-NOMIS, a South Korean company offering face-swapping and "nudify" AI services. The exposed 47.8GB dataset included nearly 100,000 files. Many depicted explicit deepfake images, some involving minors or celebrities. No personal data was found, but the breach was a serious failure in data security and consent safeguards in AI image-generation platforms.
Alleged Fraudulent Prompts via AIXBT Dashboard Led Purported AI Trading Agent to Transfer 55.5 ETH from Simulacrum Wallet
A reported hacker attack allegedly compromised the autonomous AI crypto bot AIXBT, purportedly resulting in the theft of 55.5 ETH (approximately $106,200). The attacker is reported to have infiltrated the secure dashboard of the AIXBT autonomous system at 2:00 AM UTC on March 18, 2025, and allegedly queued two fraudulent prompts that instructed the AI agent to transfer funds from its simulacrum wallet.
Alleged ChatGPT Misuse by Contractor Leads to Reported Data Exposure in New South Wales Resilient Homes Program
A former contractor of the New South Wales Reconstruction Authority reportedly uploaded a spreadsheet containing personal and health information of Resilient Homes Program applicants to ChatGPT during a three-day period in March 2025. Up to 3,000 people may have reportedly been affected.