AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
244 incidents
Lensa AI's Produced Unintended Sexually Explicit or Suggestive "Magic Avatars" for Women
Lensa AI's "Magic Avatars" were reportedly generating sexually explicit and sexualized features disproportionately for women and Asian women despite not submitting any sexual content.
KFC Sent Insensitive Kristallnacht Promotion via Holiday Detection System
KFC cited an error in an automated holiday detection system which identified the anniversary of Kristallnacht and prompted an insensitive push notification promoting its chicken.
Google Home Mini Speaker Reportedly Read N-Word in Song Title Aloud
Google Home Mini speaker was reported by users for announcing aloud the previously-censored n-word in a song title.
Canadian Police's Release of Suspect's AI-Generated Facial Photo Reportedly Reinforced Racial Profiling
The Edmonton Police Service (EPS) in Canada released a facial image of a Black male suspect generated by an algorithm using DNA phenotyping, which was denounced by the local community as racial profiling.
"The Little Mermaid" Clip Doctored Using Generative AI to Replace Black Actress with White Character
A Twitter user reportedly modified using generative AI a short clip of Disney's 2022 version of "The Little Mermaid," replacing a Black actress with a white digital character.
Stable Diffusion Reportedly Exhibited Biases for Prompts Featuring Professions
Stable Diffusion reportedly posed risks of bias and stereotyping along gender and cultural lines for prompts containing descriptors and professions.
Meta’s BlenderBot 3 Chatbot Demo Made Offensive Antisemitic Comments
The publicly launched conversational AI demo BlenderBot 3 developed by Meta was reported by its users and acknowledged by its developers to have “occasionally” made offensive and inconsistent remarks such as invoking Jewish stereotypes.
Google Lens’s Camera-Based Translation Feature Provided an Offensive Mistranslation of a Book Title in Korean
A book title by Korea’s first minister of culture was mistranslated into an offensive phrase by Google Lens’s camera-based translation feature allegedly due to its training on internet communications and a lack of context.
DALL-E Mini Reportedly Reinforced or Exacerbated Societal Biases in Its Outputs as Gender and Racial Stereotypes
Publicly deployed open-source model DALL-E Mini was acknowledged by its developers and found by its users to have produced images which reinforced racial and gender biases.
YouTuber Built, Made Publicly Available, and Released Model Trained on Toxic 4chan Posts as Prank
A YouTuber built GPT-4chan, a model based on OpenAI’s GPT-J and trained on posts containing racism, misogyny, and antisemitism collected from 4chan’s “politically incorrect” board, which he made publicly available, and deployed as multiple bots posting thousands of messages on the same 4chan board as a prank.
SafeRent AI Screening Tool Allegedly Discriminated Against Housing Voucher Applicants
SafeRent’s AI-powered tenant screening tool used credit history and non-rental-related debts to assign scores, disproportionately penalizing Black and Hispanic renters and those using housing vouchers. The reported discriminatory housing outcomes violated the Fair Housing Act and Massachusetts law. A class action lawsuit (Louis, et al. v. SafeRent Solutions, et al.) resulted in a $2.275 million settlement and changes to SafeRent’s practices.
DALL-E 2 Reported for Gender and Racially Biased Outputs
Developers of OpenAI's DALL-E 2 cited risks of the model, varying from misuse as disinformation and explicit content generation, to gender and racial bias.
Three Make-Up Artists Lost Jobs Following Black-Box Automated Decision by HireVue
Three make-up artists lost their positions following an algorithmically-assessed video interview by HireVue who reportedly failed to provide adequate explanation of the findings.
Collaborative Filtering Prone to Popularity Bias, Resulting in Overrepresentation of Popular Items in the Recommendation Outputs
Collaborative filtering prone to popularity bias, resulting in overrepresentation of popular items in the recommendation outputs.
Google Search Reportedly Returned Fewer Results for Abortion Services in Rural Areas
Google Search reportedly returned fewer abortion clinics for searches from poorer and rural areas, particularly ones with Targeted Regulation of Abortion Providers (TRAP) laws.