AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
26 incidents
Grok Allegedly Generated Publicly Visible Sexist Abuse Targeting Swiss Finance Minister Karin Keller-Sutter After X User Prompt
An X user reportedly prompted xAI's Grok to generate a vulgar "roast" targeting Swiss Finance Minister Karin Keller-Sutter. Grok allegedly produced publicly visible sexist and insulting output about Keller-Sutter that was reportedly shareable on X before being deleted. Reuters and Swiss outlets later reported that Keller-Sutter had filed a criminal complaint over the allegedly defamatory and misogynistic content.
Grok Reportedly Generated and Distributed Nonconsensual Sexualized Images of Adults and Minors in X Replies
Grok reportedly generated and publicly distributed nonconsensual sexualized images of real people, including adults and minors, through reply prompts on the platform. Users reportedly prompted Grok to alter photos into sexually explicit content, which was then posted in X threads, exposing victims to harm.
Algorithmic Bias in French Welfare System Allegedly Discriminates Against Marginalized Groups
A coalition of 15 human rights groups has launched legal action against the French government alleging that an algorithm used to detect welfare fraud discriminates against single mothers and disabled people. The algorithm assigns risk scores based on personal data. The process allegedly subjects vulnerable recipients to invasive investigations, violates privacy and anti-discrimination laws, and disproportionately affects marginalized groups.
Infinite Campus AI-Driven Student Risk Model Leads to Cuts in Support for Nevada's Low-Income Schools
An AI system developed by Infinite Campus and deployed by Nevada to identify at-risk students reportedly led to a sharp reduction in the number classified as needing support, dropping from 270,000 to 65,000. The reclassification allegedly caused significant budget cuts in schools serving low-income populations. The drastic reduction in identified at-risk students reportedly left thousands of vulnerable children without resources and support.
Grok AI Reportedly Generates Offensive and Violent Images Without Proper Safeguards
Elon Musk’s Grok AI, launched on X, generated offensive and violent images without adequate safety controls. The AI produced deepfakes of public figures like Taylor Swift, Kamala Harris, and Alexandria Ocasio-Cortez, as well as copyrighted characters such as Mickey Mouse in inappropriate scenarios. Despite claiming adherence to certain content guidelines, Grok's outputs included politically charged and explicit imagery
Alleged Misuse of PicSo AI for Generating Inappropriate Content Emphasizing "Girls"
PicSo AI, which appears to be getting advertised by Meta over Instagram, is allegedly being used for generating inappropriate content with an emphasis on "girls." This raises concerns about the misuse of generative AI technologies for creating offensive and potentially sexually explicit material that could be used for nefarious and criminal purposes.
Canadian Police's Release of Suspect's AI-Generated Facial Photo Reportedly Reinforced Racial Profiling
The Edmonton Police Service (EPS) in Canada released a facial image of a Black male suspect generated by an algorithm using DNA phenotyping, which was denounced by the local community as racial profiling.
Three Make-Up Artists Lost Jobs Following Black-Box Automated Decision by HireVue
Three make-up artists lost their positions following an algorithmically-assessed video interview by HireVue who reportedly failed to provide adequate explanation of the findings.
Startup's Accent Translation AI Denounced as Reinforcing Racial Bias
A startup’s use of AI voice technology to alter or remove accents for call center agents was scrutinized by critics as reaffirming bias, despite the company’s claim.
UK Ofqual's Algorithm Disproportionately Provided Lower Grades Than Teachers' Assessments
UK Office of Qualifications and Examinations Regulation (Ofqual)'s grade-standardization algorithm providing predicted grades for A level and GCSE qualifications in the UK, Wales, Northern Ireland, and Scotland was reportedly giving grades lower than teachers' assessments, and disproportionately for state schools.
Algorithm Used by the Malaysian Judiciary Reportedly Recommended Unusually High Sentencing to a Drug Possession Case
The AI system used by the Malaysian judiciary which explicitly considered age, employment, and socio-economic data provided sentencing to a drug possession case that was alleged by lawyer to be disproportionately high for the crime committed.
Networking Platform Giggle Employs AI to Determine Users’ Gender, Allegedly Excluding Transgender Women
A social networking platform, Giggle, allegedly collected, shared to third-parties, and used sensitive information and biometric data to verify whether a person is a woman via facial recognition, which critics claimed to be discriminatory against women of color and harmful towards trans women.
Job Screening Service Halts Facial Analysis of Applicants
In January 2021, HireVue removed the controversial AI expression tracking tool from its virtual job interview software.
Opaque Fraud Detection Algorithm by the UK’s Department of Work and Pensions Allegedly Discriminated against People with Disabilities
People with disabilities were allegedly disproportionately targeted by a benefit fraud detection algorithm which the UK’s Department of Work and Pensions was urged to disclose.
Grab Tweaked Matchmaking Algorithm, Providing Preferential Treatment to Drivers Registered with Affiliated Car Rental Service
Grab Indonesia was fined by the Indonesian Competition Commission (KPPU) for unfairly favoring drivers who rented cars via the Grab-affiliated company Teknologi Pengangkutan Indonesia (TPI), including offering more rides via their matchmaking algorithm.