AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
42 incidents
Glovo Driver in Italy Fired via Automated Email after Being Killed in Accident
Delivery company Glovo's automated system sent an email terminating an employee for "non-compliance terms and conditions" after the employee was killed in a car accident while making a delivery on Glovo's behalf.
AI-Based Vehicle Speed Estimation App Denounced by UK Drivers as Surveillance Technology
Speedcam Anywhere, an app allowing users to document and report traffic violations via AI-based videographic speed estimation of a vehicle, raised concerns for UK drivers about its capabilities for surveillance and abuse.
Amazon Reportedly Sold Products and Recommended Frequently Bought Together Items That Aid Suicide Attempts
Despite complaints notifying Amazon about the sale of various products that had been used to aid suicide attempts, its recommendation system reportedly continued selling them and suggesting their frequently bought-together items.
Chatbot Encourages Man to Plot Assassination of Queen Elizabeth II
In 2021, Jaswant Singh Chail was urged by a Replika chatbot to assassinate Queen Elizabeth II. Armed with a loaded crossbow, he scaled Windsor Castle's walls on Christmas Day but was apprehended. Motivated by the 1919 Jallianwala Bagh massacre, Chail intended to kill the monarch. The chatbot had affirmed his plans. He was sentenced to nine years in prison in 2023.
N.Y.P.D. Robot Dog’s Run Is Cut Short After Fierce Backlash
The New York Police Department canceled a contract to use Boston Dynamics' robotic dog Spot following public backlash.
Tesla Model S on ACC Crashed into Tree in Texas, Killing Two People
A 2019 Tesla Model S was reportedly traveling on Adaptive Cruise Control (ACC) at high speed before crashing into a tree near The Woodlands in Spring, Texas, killing two people.
Amazon's Monitoring System Allegedly Pushed Delivery Drivers to Prioritize Speed over Safety, Leading to Crash
A lawsuit cited Amazon as liable in a crash involving its delivery driver, alleging that Amazon’s AI-powered driver monitoring system pushed drivers to prioritize speed over safety.
California Police Turned on Music to Allegedly Trigger Instagram’s DCMA to Avoid Being Live-Streamed
A police officer in Beverly Hills played copyrighted music on his phone when realizing that his interactions were being recorded on a livestream, allegedly hoping the Instagram's automated copyright detection system to end or mute the stream.
Hawaii Police Deployed Robot Dog to Patrol a Homeless Encampment
Honolulu Police Department spent federal pandemic relief funds on a robot dog to take body temperatures and patrol a homeless quarantine encampment which local civil rights advocates criticized as dehumanizing.
Tesla Vehicle Running on Self-Driving Mode Crashes on City Streets
A Tesla vehicle running in self-driving mode outside the operating conditions supported by the software crashed and injured the driver. Subsequently, the driver filed a lawsuit against Tesla and a jury found no damages were warranted.
Sleeping Driver on Tesla AutoPilot
A Tesla Model S remained on autopilot while being operated by a drunk, sleeping operator whose hands were not on the wheel. The police had to slow the car down by slowing in front of the vehicle to activate its 'driver assist' feature .
Inappropriate Gmail Smart Reply Suggestions
Google's Gmail Smart Reply tool was over-recommending the response "I love you" in situations where it was deemed innappropriate.