AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
58 incidents
Unrestricted AI Avatar Tool Accidentally Released by TikTok Permits Recitation of Hitler Quotes and Other Harmful Speech
TikTok accidentally released an internal version of its AI digital avatar tool without safeguards, allowing users to generate videos where avatars could recite harmful content, including quotes from Hitler. The tool, meant for advertisers, was accessible to personal accounts and lacked the watermark indicating AI-generated content. TikTok has since removed the tool and acknowledged the problem.
VA Suicide Prevention Algorithm REACH VET Reportedly Prioritizes Men Over Women Veterans
An AI program named REACH VET, designed and used by the Department of Veterans Affairs (VA) to prevent veteran suicides, was reportedly found to prioritize white men while neglecting female veterans and survivors of military sexual trauma. This oversight persists despite rising suicide rates among these groups. The incident is an example of algorithmic bias and the exclusion of critical risk factors for female veterans.
GPT-4o's Chinese Tokens Reportedly Compromised by Spam and Pornography Due to Inadequate Filtering
OpenAI's GPT-4o was found to have its Chinese token training data compromised by spam and pornographic phrases due to inadequate data cleaning. Tianle Cai, a Ph.D. student at Princeton University, identified that most of the longest Chinese tokens were irrelevant and inappropriate, primarily originating from spam and pornography websites. The polluted tokens could lead to hallucinations, poor performance, and potential misuse, undermining the chatbot's reliability and safety measures.
Proliferation of Products on Amazon Titled with ChatGPT Error Messages
Products named after ChatGPT error messages are proliferating on Amazon, such as lawn chairs and religious texts. These names, often resembling AI-generated errors, indicate a lack of editing and undermine the sense of authenticity and reliability of product listings.
Child Sexual Abuse Material Taints Image Generators
The LAION-5B dataset (a commonly used dataset with more than 5 billion image-description pairs) was found by researchers to contain child sexual abuse material (CSAM), which increases the likelihood that downstream models will produce CSAM imagery. The discovery taints models built with the LAION dataset requiring many organizations to retrain those models. Additionally, LAION must now scrub the dataset of the imagery.
Reported Accidental Exposure of 38TB of Data by Microsoft's AI Research Team
Microsoft's AI research team reportedly accidentally exposed 38TB of sensitive data while publishing open-source training material on GitHub. The exposure allegedly included secrets, private keys, passwords, and internal Microsoft Teams messages. The team reportedly utilized Azure's Shared Access Signature (SAS) tokens for sharing, which were purportedly misconfigured, leading to the wide exposure of data.
Texas A&M-Commerce Professor Reportedly Used ChatGPT to Accuse Students of Cheating and Assign Incomplete Grades
On May 15, 2023, Texas A&M University-Commerce instructor Jared Mumm reportedly emailed animal-science seniors that he had copied their final assignments into ChatGPT and would assign zeros or temporary incompletes when the chatbot claimed the text was AI-generated. Students said the accusations caused panic and put graduation or diplomas at risk; the university later said no students failed or were barred from graduating because of the issue.
ChatGPT Reportedly Implicated in Samsung Data Leak of Source Code and Meeting Notes
Samsung engineers are reported to have inadvertently leaked sensitive company data sometime in March 2023, including source code and internal meeting notes, by using ChatGPT to assist with tasks. ChatGPT allegedly retained the inputted data, leading to a purported breach of confidentiality.
Delivery Robot Rolled Through Crime Scene
A Serve Robotics delivery robot was shown on video rolling through a crime scene blocked off by police tape.
Private Medical Photos Were Reportedly Found in LAION-5B AI Training Dataset
In September 2022, an artist using the name Lapine reported finding private post-operative medical photos of herself in LAION-5B, a web-scraped image-text dataset used in AI image-synthesis research. Ars Technica confirmed her images were referenced in the dataset and reported finding thousands of similar patient medical-record photos.
AI-Based Vehicle Speed Estimation App Denounced by UK Drivers as Surveillance Technology
Speedcam Anywhere, an app allowing users to document and report traffic violations via AI-based videographic speed estimation of a vehicle, raised concerns for UK drivers about its capabilities for surveillance and abuse.
BMW Sedan Made a Prohibited Left Turn, Colliding with a Cruise Autonomous Vehicle
A BMW Sedan reportedly made an illegal left turn, causing a minor collision but no injuries with a Cruise autonomous vehicle (AV) operating in autonomous mode.
AI Tools Failed to Sufficiently Predict COVID Patients, Some Potentially Harmful
AI tools failed to sufficiently predict COVID patients, some potentially harmful.
N.Y.P.D. Robot Dog’s Run Is Cut Short After Fierce Backlash
The New York Police Department canceled a contract to use Boston Dynamics' robotic dog Spot following public backlash.
Tesla Model S on ACC Crashed into Tree in Texas, Killing Two People
A 2019 Tesla Model S was reportedly traveling on Adaptive Cruise Control (ACC) at high speed before crashing into a tree near The Woodlands in Spring, Texas, killing two people.