AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
217 incidents
Google Home Mini Speaker Reportedly Read N-Word in Song Title Aloud
Google Home Mini speaker was reported by users for announcing aloud the previously-censored n-word in a song title.
Canadian Police's Release of Suspect's AI-Generated Facial Photo Reportedly Reinforced Racial Profiling
The Edmonton Police Service (EPS) in Canada released a facial image of a Black male suspect generated by an algorithm using DNA phenotyping, which was denounced by the local community as racial profiling.
Stable Diffusion Reportedly Exhibited Biases for Prompts Featuring Professions
Stable Diffusion reportedly posed risks of bias and stereotyping along gender and cultural lines for prompts containing descriptors and professions.
Meta’s BlenderBot 3 Chatbot Demo Made Offensive Antisemitic Comments
The publicly launched conversational AI demo BlenderBot 3 developed by Meta was reported by its users and acknowledged by its developers to have “occasionally” made offensive and inconsistent remarks such as invoking Jewish stereotypes.
Google Lens’s Camera-Based Translation Feature Provided an Offensive Mistranslation of a Book Title in Korean
A book title by Korea’s first minister of culture was mistranslated into an offensive phrase by Google Lens’s camera-based translation feature allegedly due to its training on internet communications and a lack of context.
DALL-E Mini Reportedly Reinforced or Exacerbated Societal Biases in Its Outputs as Gender and Racial Stereotypes
Publicly deployed open-source model DALL-E Mini was acknowledged by its developers and found by its users to have produced images which reinforced racial and gender biases.
SafeRent AI Screening Tool Allegedly Discriminated Against Housing Voucher Applicants
SafeRent’s AI-powered tenant screening tool used credit history and non-rental-related debts to assign scores, disproportionately penalizing Black and Hispanic renters and those using housing vouchers. The reported discriminatory housing outcomes violated the Fair Housing Act and Massachusetts law. A class action lawsuit (Louis, et al. v. SafeRent Solutions, et al.) resulted in a $2.275 million settlement and changes to SafeRent’s practices.
DALL-E 2 Reported for Gender and Racially Biased Outputs
Developers of OpenAI's DALL-E 2 cited risks of the model, varying from misuse as disinformation and explicit content generation, to gender and racial bias.
Three Make-Up Artists Lost Jobs Following Black-Box Automated Decision by HireVue
Three make-up artists lost their positions following an algorithmically-assessed video interview by HireVue who reportedly failed to provide adequate explanation of the findings.
Collaborative Filtering Prone to Popularity Bias, Resulting in Overrepresentation of Popular Items in the Recommendation Outputs
Collaborative filtering prone to popularity bias, resulting in overrepresentation of popular items in the recommendation outputs.
Google Search Reportedly Returned Fewer Results for Abortion Services in Rural Areas
Google Search reportedly returned fewer abortion clinics for searches from poorer and rural areas, particularly ones with Targeted Regulation of Abortion Providers (TRAP) laws.
Honorlock's AI Proctoring System Reportedly Led Broward College to Accuse a Teenager of Cheating
During a February 2022 online biology exam, Honorlock reportedly flagged a Florida teenager taking a Broward College class as suspicious. The student denied cheating, but Broward College allegedly relied on the remote-proctoring system's flag in pursuing an academic-dishonesty accusation that the student said was unfounded.
Justice Department’s Recidivism Risk Algorithm PATTERN Allegedly Caused Persistent Disparities Along Racial Lines
Department of Justice’s inmate-recidivism risk assessment tool was reported to have produced racially uneven results, misclassifying risk levels for inmates of color.
TikTok's "For You" Algorithm Allegedly Abused by Online Personality to Promote Anti-Women Hate
TikTok’s “For You” algorithm allegedly boosted or was manipulated by an online personality to artificially boost his content which promotes extreme misogynistic views towards teenagers and men, despite breaking its rules.
Alexa Recommended Dangerous TikTok Challenge to Ten-Year-Old Girl
Amazon’s voice assistant Alexa suggested “the penny challenge,” which involves dangerously touching a coin to the prongs of a half-exposed plug, when a ten-year-old girl asked for a challenge to do.