AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
961 incidents
AI-Generated Obituaries Are Reportedly Intensifying Grief for Bereaved Families
AI-generated obituaries on various websites are reported to have compounded the grief of bereaved families by spreading incorrect and unauthorized information about their loved ones. These obituaries, produced without the families' knowledge, often contain errors and appear on ad-filled sites, exacerbating the emotional distress of the grieving process.
Baidu Robotaxi Allegedly Involved in Collision with Pedestrian in Wuhan
On July 7, 2024, a Baidu Robotaxi reportedly collided with a pedestrian at a traffic intersection in Wuhan. The incident occurred as the autonomous vehicle started moving on a green light while the pedestrian was allegedly crossing against a red light. The pedestrian sustained minor injuries, and Baidu was reported to have been cooperating with local authorities for further investigation.
Figma Disables AI Feature After Accusations of Copying Apple’s Weather App
Figma has temporarily disabled its AI design feature, "Make Design," after accusations of copying Apple’s Weather app. Andy Allen of NotBoring Software highlighted the issue, prompting Figma CEO Dylan Field to deny claims of training the AI on specific app designs. However, Field acknowledged flaws in the QA process and promised to suspend the feature until it meets quality standards. The incident has implications for designers.
Female Politicians in the United Kingdom Reportedly Victimized by Purported Deepfake Pornography
Female politicians in the United Kingdom, including Angela Rayner, Gillian Keegan, Penny Mordaunt, Priti Patel, Stella Creasy, and Dehenna Davison, have reportedly been targeted by non-consensual, purportedly AI-generated deepfake pornography. The images, which some accounts suggest may have circulated online for an extended period, were reported to have caused distress and, in at least some instances, to have prompted reports to or involvement by law enforcement.
OpenAI's ChatGPT Mac App Stored User Data in Unencrypted Text Files
OpenAI's ChatGPT macOS app stored user conversations in plain text. If accessed by a malicious actor, these conversations could have been easily read. The critical security flaw was demonstrated by a third party and ultimately resolved after OpenAI released an update to encrypt the stored data.
Students at Australian Schools Reportedly Used AI Nudification Tools to Create Nonconsensual Sexual Images of Classmates and Teachers
Throughout 2024, schools in Australia reportedly dealt with a significant rise and proliferation of nonconsensual deepfake pornography of students. Often, male students are reported to use "nudify" apps such as Undress AI with images of their classmates and teachers. Many of the sites have reportedly remained legal and allegedly accessible to minors, who in turn are purportedly using the sites to generate pornography of their peers.
AI Work Assistants Require More Effort Than Expected, CIOs Say
AI work assistants, such as Copilot for Microsoft 365 and Gemini for Google Workspace, are proving to be more labor-intensive than anticipated for enterprises. CIOs report that these AI tools struggle with outdated or inaccurate data, leading to incorrect outputs. Companies are finding they must invest heavily in data management to ensure reliability. This added effort has led to delays in deployment and frustration, as businesses work to maximize the potential of these expensive AI tools.
Department for Work and Pensions (DWP) Algorithm Wrongly Flags 200,000 for Housing Benefit Fraud
A Department for Work and Pensions (DWP) algorithm wrongly flagged over 200,000 UK housing benefit claims as high risk, resulting in unnecessary investigations. Two-thirds of these flagged claims were legitimate, causing wasted public funds and stress for claimants. Despite initial success in a pilot, the algorithm's real-world performance fell short. This incident highlights the risks of overreliance on automated systems in welfare administration.
Senior Minister of Singapore Lee Hsien Loong Allegedly Misrepresented in Two Deepfake Videos on Foreign Relations
Two deepfake videos falsely depicted Senior Minister of Singapore Lee Hsien Loong commenting on US-China relations and the South China Sea. The videos misleadingly attributed views to him and the Singapore government. Posted on TikTok, the videos amassed over 190,000 views before being removed.
Instagram's Algorithm Reportedly Recommended Sexual Content to Teenagers' Accounts
Tests by The Wall Street Journal and a researcher reportedly revealed that Instagram's AI-driven Reels algorithm recommends sexually suggestive content to accounts listed as 13 years old. Despite Meta's commitment to restrict such content for minors, explicit videos were served within minutes of account creation, according to the findings.
Erroneous Declined Transaction Notification by PayPal AI Assistant
On July 13th, 2024, a user reported an incident involving PayPal's generative AI chatbot. The chatbot allegedly incorrectly informed the user of a declined transaction that never occurred, causing confusion and prompting a call to customer service for clarification. This false alert suggests a flaw in the AI system's reliability. The incident created unnecessary labor for both the user and PayPal's human support, demonstrating the potential harm of deploying generative AI without thorough testing and error handling mechanisms.
Leading AI Models Reportedly Found to Mimic Russian Disinformation in 33% of Cases and to Cite Fake Moscow News Sites
An audit by NewsGuard revealed that leading chatbots, including ChatGPT-4, You.com’s Smart Assistant, and others, repeated Russian disinformation narratives in one-third of their responses. These narratives originated from a network of fake news sites created by John Mark Dougan (Incident 701). The audit tested 570 prompts across 10 AI chatbots, showing that AI remains a tool for spreading disinformation despite efforts to prevent misuse.
Tesla Reportedly in Autopilot Mode Hits Parked Police Vehicle in Fullerton, California
A Tesla reportedly in self-driving mode crashed into a parked patrol vehicle in Fullerton, California while the officer was responding to a fatal DUI crash. The officer narrowly escaped injury. The driver reports having been distracted by a cellphone and having relied on the Tesla’s AI. (The earlier crash involved a suspected DUI driver who killed a motorcyclist stopped at a red light.)
Purportedly AI-Generated Papers Are Alleged to Have Manipulated Scopus Rankings in Top Philosophy Journals
Three reportedly fake journals published by Addleton Academic Publishers allegedly manipulated Scopus rankings by extensively cross-citing each other and using AI-generated papers filled with buzzwords. These journals, placed in the top 10 of Scopus's 2023 CiteScore philosophy list, reportedly featured fake authors, affiliations, and grant numbers, in turn allegedly pushing legitimate journals to lower tiers, affecting academic evaluations and awards.
Independent News Sites Flagged as Spam by Facebook's AI Moderation System
Facebook's AI moderation system mistakenly flagged and removed legitimate news articles from independent local publishers as spam. This affected publishers worldwide, disrupting content distribution and impacting traffic. Despite attempts to appeal, many publishers received no response.