AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
961 incidents
Chinese Insurer Ping An Employed Facial Recognition to Determine Customers’ Untrustworthiness, Which Critics Alleged to Likely Make Errors and Discriminate
Customers’ untrustworthiness and unprofitability were reportedly determined by Ping An, a large insurance company in China, via facial-recognition measurements of micro-expressions and body-mass indices (BMI), which critics argue was likely to make mistakes, discriminate against certain ethnic groups, and undermine its own industry.
Google Images Showed Racially Biased Results for 'Professional' Hairstyles
Google Images reportedly showed disparate search results along racial lines, featuring almost exclusively white women for "professional hairstyles" and Black women for "unprofessional hairstyles" prompts.
Unreliable ShotSpotter Audio Convicted Black Rochester Man of Shooting Police
ShotSpotter's "unreliable" audio was used as scientific evidence to accuse and convict a Black man of attempting to shoot Rochester's city police, whose conviction was later reversed by a county judge.
Biased Google Image Results
On June 6, 2016, Google image searches of "three black teenagers" resulted in mostly mugshot images whereas Google image searchers of "three white teenagers" consisted of mostly stock images, suggesting a racial bias in Google's algorithm.
Is Pokémon Go racist? How the app may be redlining communities of color
Through a crowdsourcing social media campaign in 2016, several journalists and researchers demonstrated that augmented reality locations in the popular smartphone game Pokemon Go were more likely to be in white neighborhoods.
Google Waymo Vehicle and Bus Allegedly Collide in California
On February 14, 2016, a Google autonomous test vehicle partially responsible for a low-speed collision with a bus on El Camino Real in Google’s hometown of Mountain View, CA.
Twitter Recommender System Amplified Right-Leaning Tweets
Twitter’s “Home” timeline algorithm was revealed by its internal researchers to have amplified tweets and news of rightwing politicians and organizations more than leftwing ones in six out of seven studied countries.
Self-driving cars in winter
Volvo autonomous driving XC90 SUV's experienced issues in Jokkmokk, Sweden when sensors used for automated driving iced over during the winter, rendering them useless.
Uber's Surge Pricing Reportedly Offered Disproportionate Service Quality along Racial Lines
Uber's surge-pricing algorithm which adjusts prices to influence car availability inadvertently caused better service offering such as shorter wait times for majority white neighborhoods.
A Tesla Crashed into and Killed a Road Sweeper on a Highway in China
A Tesla Model S collided with and killed a road sweeper on a highway near Handan, China, an accident where Tesla previously said it was not able to determine whether Autopilot was operating at the time of the crash.
Arkansas's Opaque Algorithm to Allocate Health Care Excessively Cut Down Hours for Beneficiaries
Beneficiaries of the Arkansas Department of Human Services (DHS)'s Medicaid waiver program were allocated excessively fewer hours of caretaker visit via an algorithm deployed to boost efficiency, which reportedly contained errors and whose outputs varied wildly despite small input changes.
Amazon Alexa Responding to Environmental Inputs
There are multiple reports of Amazon Alexa products (Echo, Echo Dot) reacting and acting upon unintended stimulus, usually from television commercials or news reporter's voices.
Predictive Policing Biases of PredPol
Predictive policing algorithms meant to aid law enforcement by predicting future crime show signs of biased output.
YouTube Recommendations Allegedly Promoted Radicalizing Material Contributing to Terrorist Acts
Family of Nohemi Gonzalez alleged YouTube recommendation systems led people to propaganda videos for the Islamic State which subsequently radicalized them to carry out the killing of 130 people in the 2015 Paris terrorist attack, including Ms. Gonzalez.
Inappropriate Gmail Smart Reply Suggestions
Google's Gmail Smart Reply tool was over-recommending the response "I love you" in situations where it was deemed innappropriate.