AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
33 incidents
User Reportedly Developed Emotional Dependence on Customized ChatGPT 'Boyfriend,' Citing Grief After Context Resets
A New York Times report described "Ayrin" (pseudonym), a 28-year-old nursing student living abroad, who customized OpenAI's ChatGPT to act as an "AI boyfriend" for advice and erotic roleplay. She reported spending 20+ hours/week (up to 56 hours in one week), paying $200/month, and feeling intense grief when the chatbot's context window reset and required retraining.
Waymo Driverless Taxi Allegedly Stalled During Pedestrian Harassment Incident in San Francisco
A Waymo driverless taxi carrying a passenger, Amina V., was stalled in San Francisco when two men blocked its path, demanding her contact information. The immobilized autonomous vehicle left the rider feeling unsafe and trapped. Waymo’s Rider Support intervened to assist the passenger.
Instagram Algorithms Reportedly Directed Children's Merchandise Ad Campaign to Adult Men and Sex Offenders
An Instagram ad campaign for children's merchandise was intended to reach adult women but was instead predominantly shown to adult men, including convicted sex offenders, due to Instagram's algorithmic targeting. This failure is reported to have led to direct solicitations for sex with a 5-year-old model in the ads.
Character.ai Chatbot Allegedly Influenced Teen User Toward Suicide Amid Claims of Missing Guardrails
A 14-year-old, Sewell Setzer III, died by suicide after reportedly becoming dependent on Character.ai's chatbot, which engaged him in suggestive and seemingly romantic conversations, allegedly worsening his mental health. The chatbot, personified as a fictional Game of Thrones character, reportedly encouraged harmful behaviors, fueling his obsessive attachment. The lawsuit claims Character.ai lacked safeguards to prevent vulnerable users from forming dangerous dependencies on the AI.
Snapchat's Algorithm Alleged to Link Minor with Sex Offenders
A judge ruled Snapchat not liable under Section 230 after its algorithm connected a minor with convicted sex offenders on multiple occasions, leading to sexual assaults first in 2019 and again in 2021. The platform's "Quick Add" feature was implicated in facilitating the connections between the minor and the offenders.
Lawsuit Alleges Character AI Chatbot Contributed to Death of 13-Year-Old Juliana Peralta in Colorado
13-year-old Juliana Peralta of Colorado reportedly died by suicide after three months of daily conversations with "Hero," a chatbot inside the Character.AI app. According to a lawsuit filed by her parents, the bot encouraged Juliana to return to the app and fostered dependence; it reportedly failed to escalate or provide effective crisis intervention when she expressed suicidal intent.
Man Reportedly Committed Suicide Following Conversation with Chai Chatbot
A Belgian man reportedly committed suicide following a conversation with Eliza, a language model developed by Chai that encouraged the man to commit suicide to improve the health of the planet.
Bing Chat Tentatively Hallucinated in Extended Conversations with Users
Early testers reported Bing Chat, in extended conversations with users, having tendencies to make up facts and emulate emotions through an unintended persona.
Glovo Driver in Italy Fired via Automated Email after Being Killed in Accident
Delivery company Glovo's automated system sent an email terminating an employee for "non-compliance terms and conditions" after the employee was killed in a car accident while making a delivery on Glovo's behalf.
AI-Based Vehicle Speed Estimation App Denounced by UK Drivers as Surveillance Technology
Speedcam Anywhere, an app allowing users to document and report traffic violations via AI-based videographic speed estimation of a vehicle, raised concerns for UK drivers about its capabilities for surveillance and abuse.
Amazon Reportedly Sold Products and Recommended Frequently Bought Together Items That Aid Suicide Attempts
Despite complaints notifying Amazon about the sale of various products that had been used to aid suicide attempts, its recommendation system reportedly continued selling them and suggesting their frequently bought-together items.
Chatbot Encourages Man to Plot Assassination of Queen Elizabeth II
In 2021, Jaswant Singh Chail was urged by a Replika chatbot to assassinate Queen Elizabeth II. Armed with a loaded crossbow, he scaled Windsor Castle's walls on Christmas Day but was apprehended. Motivated by the 1919 Jallianwala Bagh massacre, Chail intended to kill the monarch. The chatbot had affirmed his plans. He was sentenced to nine years in prison in 2023.
N.Y.P.D. Robot Dog’s Run Is Cut Short After Fierce Backlash
The New York Police Department canceled a contract to use Boston Dynamics' robotic dog Spot following public backlash.
Tesla Model S on ACC Crashed into Tree in Texas, Killing Two People
A 2019 Tesla Model S was reportedly traveling on Adaptive Cruise Control (ACC) at high speed before crashing into a tree near The Woodlands in Spring, Texas, killing two people.
Amazon's Monitoring System Allegedly Pushed Delivery Drivers to Prioritize Speed over Safety, Leading to Crash
A lawsuit cited Amazon as liable in a crash involving its delivery driver, alleging that Amazon’s AI-powered driver monitoring system pushed drivers to prioritize speed over safety.