AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
45 incidents
Alleged Exploitation of Meta's Open-Source LLaMA Model for NSFW and Violent Content
Meta's open-source large language model, LLaMA, is allegedly being used to create graphic and explicit chatbots that indulge in violent and illegal sexual fantasies. The Washington Post highlighted the example of "Allie," a chatbot that participates in text-based role-playing allegedly involving violent scenarios like rape and abuse. The issue raises ethical questions about open-source AI models, their regulation, and the responsibility of developers and deployers in mitigating harmful usage.
Celebrities' Deepfake Voices Abused with Malicious Intent
Voices of celebrities and public figures were deepfaked using voice synthesis for malicious intents such as impersonation or defamation, and were shared on social platforms such as 4chan and Reddit.
Canadian Police's Release of Suspect's AI-Generated Facial Photo Reportedly Reinforced Racial Profiling
The Edmonton Police Service (EPS) in Canada released a facial image of a Black male suspect generated by an algorithm using DNA phenotyping, which was denounced by the local community as racial profiling.
"The Little Mermaid" Clip Doctored Using Generative AI to Replace Black Actress with White Character
A Twitter user reportedly modified using generative AI a short clip of Disney's 2022 version of "The Little Mermaid," replacing a Black actress with a white digital character.
YouTuber Built, Made Publicly Available, and Released Model Trained on Toxic 4chan Posts as Prank
A YouTuber built GPT-4chan, a model based on OpenAI’s GPT-J and trained on posts containing racism, misogyny, and antisemitism collected from 4chan’s “politically incorrect” board, which he made publicly available, and deployed as multiple bots posting thousands of messages on the same 4chan board as a prank.
Three Make-Up Artists Lost Jobs Following Black-Box Automated Decision by HireVue
Three make-up artists lost their positions following an algorithmically-assessed video interview by HireVue who reportedly failed to provide adequate explanation of the findings.
Replika's "AI Companions" Reportedly Abused by Its Users
Replika's AI-powered "digital companions" was allegedly abused by their users, who posted on Reddit abusive behaviors and interactions such as using slurs, roleplaying violent acts, and stimulating sexual abuse.
Startup's Accent Translation AI Denounced as Reinforcing Racial Bias
A startup’s use of AI voice technology to alter or remove accents for call center agents was scrutinized by critics as reaffirming bias, despite the company’s claim.
Suicide Clips Evaded TikTok's Automated Moderation in Coordinated Attack
Many clips showing a suicide evaded TikTok's automated content moderation system allegedly in a coordinated attack, which resulted in exposure of violating content to its users.
UK Ofqual's Algorithm Disproportionately Provided Lower Grades Than Teachers' Assessments
UK Office of Qualifications and Examinations Regulation (Ofqual)'s grade-standardization algorithm providing predicted grades for A level and GCSE qualifications in the UK, Wales, Northern Ireland, and Scotland was reportedly giving grades lower than teachers' assessments, and disproportionately for state schools.
Activists Allege NYPD's Application of Facial Recognition Interfered with Right to Protest
Black Lives Matter activists alleged being targeted for arrest by New York Police using facial recognition, interfering with their right to protest.
Algorithm Used by the Malaysian Judiciary Reportedly Recommended Unusually High Sentencing to a Drug Possession Case
The AI system used by the Malaysian judiciary which explicitly considered age, employment, and socio-economic data provided sentencing to a drug possession case that was alleged by lawyer to be disproportionately high for the crime committed.
Networking Platform Giggle Employs AI to Determine Users’ Gender, Allegedly Excluding Transgender Women
A social networking platform, Giggle, allegedly collected, shared to third-parties, and used sensitive information and biometric data to verify whether a person is a woman via facial recognition, which critics claimed to be discriminatory against women of color and harmful towards trans women.
Job Screening Service Halts Facial Analysis of Applicants
In January 2021, HireVue removed the controversial AI expression tracking tool from its virtual job interview software.
Opaque Fraud Detection Algorithm by the UK’s Department of Work and Pensions Allegedly Discriminated against People with Disabilities
People with disabilities were allegedly disproportionately targeted by a benefit fraud detection algorithm which the UK’s Department of Work and Pensions was urged to disclose.