AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
961 incidents
Startup's Accent Translation AI Denounced as Reinforcing Racial Bias
A startup’s use of AI voice technology to alter or remove accents for call center agents was scrutinized by critics as reaffirming bias, despite the company’s claim.
Xsolla Employees Fired by CEO Allegedly via Big Data Analytics of Work Activities
Xsolla CEO fired more than a hundred employees from his company in Perm, Russia, based on big data analysis of their remote digitized-work activity, which critics said was violating employee's privacy, outdated, and extremely ineffective.
Epic Systems’s Sepsis Prediction Algorithms Revealed to Have High Error Rates on Seriously Ill Patients
Epic System's sepsis prediction algorithms was shown by investigators at the University of Michigan Hospital to have high rates of false positives and false negatives, allegedly delivering inaccurate and irrelevant information on patients, contrasting sharply with their published claims.
Tesla's Autopilot Misidentified the Moon as Yellow Stop Light
Tesla's Autopilot was shown on video by its owner mistaking the moon for a yellow stop light, allegedly causing the vehicle to keep slowing down.
Facebook's Automated Moderation Flagged Gardening Group's Language Use by Mistake
Facebook's automated system flagged gardening groups' use of "hoe" and violent language against bugs as a violation by mistake.
Three Robots Collided, Sparking Fire in a Grocer's Warehouse in UK
A collision involving three robots at an Ocado's warehouse in Erith, UK, resulting in a fire but no reports of injuries.
SoftBank's Humanoid Robot, Pepper, Reportedly Frequently Made Errors, Prompting Dismissal
SoftBank's robot allegedly kept making mechanical errors, taking unplanned breaks, failing to recognize previously-met people, and breaking down during practice runs.
Facebook, Instagram, and Twitter Failed to Proactively Remove Targeted Racist Remarks via Automated Systems
Facebook's, Instagram's, and Twitter's automated content moderation failed to proactively remove racist remarks and posts directing at Black football players after finals loss, allegedly largely relying on user reports of harassment.
Skating Rink’s Facial Recognition Cameras Misidentified Black Teenager as Banned Troublemaker
A Black teenager living in Livonia, Michigan was incorrectly stopped from entering a roller skating rink after its facial-recognition cameras misidentified her as another person who had been previously banned for starting a skirmish with other skaters.
Algorithmic Allocation of Resources in Healthcare for Disabled and Elderly Care Services Allegedly Harmed Patients
A healthcare algorithm designed to equitably distribute caregiving resources is alleged to have drastically cut care hours for the disabled and elderly, purportedly leading to significant hardships and harm. Initially developed for fair resource allocation, the system reportedly faced legal challenges for its inability to accurately assess individual needs, resulting in allegedly reduced essential care.
Hiring Algorithms Provided Invalid Positive Results for Interview Responses in German
Two AI interview softwares provided positive but invalid results such as "competent" English proficiency and high match percentage for interview responses given in German by reporters.
GitHub Copilot, Copyright Infringement and Open Source Licensing
Users of GitHub Copilot can produce source code subject to license requirements without attributing and licensing the code to the rights holder.
Maryland Police Allegedly Relied on Facial Recognition Lead in Wrongful Arrest and Detention of Kimberlee Williams
Kimberlee Williams was reportedly arrested in Oklahoma on Maryland warrants after police allegedly relied on a facial recognition lead that incorrectly identified her as a suspect in bank fraud cases. The ACLU said Williams had never been to Maryland, police failed to investigate alibi evidence placing her in Oklahoma, and she spent about six months in jail before the charges were dropped.
State Farm Allegedly Discriminated against Black Customers in Claim Payout
State Farm's automated claims processing method was alleged in a class action lawsuit to have disproportionately against Black policyholders when paying out insurance claims.
Kannada Insulted by Google's Featured Answer as "Ugliest Language in India"
Google's knowledge-graph-powered algorithm showed Kannada in its featured Answer Box when prompted "ugliest language in India," causing outrage from Kannada-speaking people and government.