AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
289 incidents
NarxCare’s Risk Score Model Allegedly Lacked Validation and Trained on Data with High Risk of Bias
NarxCare's overdose risk algorithm, lacking peer-reviewed validation, uses sensitive data like doctor visits, prescriptions, and possibly genetic information, leading to significant biases against women and Black patients. Factors like sexual abuse and criminal records exacerbate stigmas and disparities, often resulting in unjust denial of necessary pain medication. The newly approved AvertD genetic test shares similar issues, further complicating and potentially harming medical treatment decisions.
Facebook’s Political Ad Detection Reportedly Showed High and Geographically Uneven Error Rates
The performance of Facebook’s political ad detection was revealed by researchers to be imprecise, uneven across countries in errors, and inadequate for preventing systematic violations of political advertising policies.
YouTube's AI Mistakenly Banned Chess Channel over Chess Language Misinterpretation
YouTube's AI-powered hate speech detection system falsely flagged chess content and banned chess creators allegedly due to its misinterpretation of strategy language such as "black," "white," and "attack" as harmful and dangerous.
AI translation is jeopardizing Afghan asylum claims
A Pashto-speaking refugee's asylum claim was denied by a US agency for a discrepancy between oral and written recount of an event allegedly due to an error of their automated translation tool which swapped pronouns of her written statement from "I" to "we".
Starship Delivery Robot Scuffed Bumper of a Resident’s Car in Texas, Allegedly Refusing to Release Footage of the Accident
A Starship food delivery robot crashed into the front bumper of a vehicle waiting at a stoplight intersection in Frisco, Texas, the video of which the company reportedly refused to release.
Tesla on Autopilot Crashed into Flipped Truck on Taiwan Highway
On a highway in Taiwan, a Tesla Sedan, reportedly operating on Autopilot mode, crashed into a large overturned truck, barely missing a pedestrian.
Unreliable ShotSpotter Audio Previously Used to Convict Chicago Man in Murder Case
ShotSpotter audios were previously admitted to convict an innocent Black man in a murder case in Chicago, resulted in his nearly-one-year-long arrest before being dismissed by prosecutors as insufficient evidence.
YouTube Auto-Moderation Reportedly Banned Women of Sex Tech Conference
YouTube’s automated content moderation tool erroneously removed The Women of Sex Tech conference's live-streamed event and banned the conference from the platform, despite not violating the platform’s sexual content policies.
Walmart's Bagging-Detection False Positives Exposed Workers to Health Risk
Walmart's theft-deterring bagging-detection system allegedly exposed workers to health risks during the coronavirus pandemic when its false positives prompted workers to unnecessarily step in to resolve the issue.
Facebook Content Moderators Demand Better Working Conditions Due to Allegedly Inadequate AI Content Moderation
Content moderators and employees at Facebook demand better working conditions, as automated content moderation system allegedly failed to achieve sufficient performance and exposed human reviewers to psychologically hazardous content such as graphic violence and child abuse.
Bug in Facebook’s Anti-Spam Filter Allegedly Blocked Legitimate Posts about COVID-19
Facebook was reported by users for blocking posts of legitimate news about the coronavirus pandemic, allegedly due to a bug in an anti-spam system.
Permanent Removal of Social Media Content via Automated Tools Allegedly Prevented Investigative Efforts
Automated permanent removal of violating social media content, such as terrorism, violent extremism, and hate speech, without archival has allegedly hindered the potential use of this content for investigating serious crimes and hampered efforts in criminal accountability.
EasyMile Self-Driving Shuttle Unexpectedly Stopped Mid-Route, Injuring a Passenger
A self-driving shuttle deployed by Smart Columbus in Linden neighborhood unexpectedly stopped on the street, which caused a woman to fall onto the floor from her seat.
Facebook Gave Vulgar English Translation of Chinese President's Name
Facebook provided a vulgar Burmese-English translation of the Chinese president's name in posts of an official Burmese politician's Facebook page announcing his visit.
COVID-19 Detection and Prognostication Models Allegedly Flagged for Methodological Flaws and Underlying Biases
Peer-review of papers about COVID-19 detection and prognostication algorithms from 2020, including deployed models, revealed none to be ready for clinical use, due to methodological flaws and underlying biases such as lacking external validation or not specifying data sources and model training details.