AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
45 incidents
Grab Tweaked Matchmaking Algorithm, Providing Preferential Treatment to Drivers Registered with Affiliated Car Rental Service
Grab Indonesia was fined by the Indonesian Competition Commission (KPPU) for unfairly favoring drivers who rented cars via the Grab-affiliated company Teknologi Pengangkutan Indonesia (TPI), including offering more rides via their matchmaking algorithm.
Chinese Tech Firms Allegedly Developed Facial Recognition to Identify People by Race, Targeting Uyghur Muslims
Various Chinese firms were revealed by patent applications to have developed facial recognition capable of detecting people by race, which critics feared would enable persecution and discrimination of Uyghur Muslims.
Argentinian City Government Deployed Teenage-Pregnancy Predictive Algorithm Using Invasive Demographic Data
In 2018, during the abortion-decriminalization debate in Argentina, the Salta city government deployed a teenage-pregnancy predictive algorithm built by Microsoft that allegedly lacked a defined purpose, explicitly considered sensitive information such as disability and whether their home had access to hot water.
Man Arrested For Sock Theft by False Facial Match Despite Alibi
A man was arrested for theft of socks from a TJ Maxx store under the guise of an eyewitness ID case, after the local police asked the store's security guard to confirm the facial recognition match produced using surveillance footage, despite him having an alibi at the time of the theft.
"Jewish Baby Strollers" Provided Anti-Semitic Google Images, Allegedly Resulting from Hate Speech Campaign
Google's Image search for "Jewish baby strollers" showed offensive, anti-Semitic results, allegedly a result of a coordinated hate-speech campaign involving malicious actors on 4chan.
Airbnb's Trustworthiness Algorithm Allegedly Banned Users without Explanation, and Discriminated against Sex Workers
Airbnb allegedly considered publicly available data on users to gauge their trustworthiness via algorithmic assessment of personality and behavioral traits, resulting in unexplained bans and discriminatory bans against sex workers.
NYPD's Deployment of Facial Recognition Cameras Reportedly Reinforced Biased Policing
New York Police Department’s use of facial recognition deployment of surveillance cameras were shown using crowdsourced volunteer data reinforcing discriminatory policing against minority communities.
Chinese Insurer Ping An Employed Facial Recognition to Determine Customers’ Untrustworthiness, Which Critics Alleged to Likely Make Errors and Discriminate
Customers’ untrustworthiness and unprofitability were reportedly determined by Ping An, a large insurance company in China, via facial-recognition measurements of micro-expressions and body-mass indices (BMI), which critics argue was likely to make mistakes, discriminate against certain ethnic groups, and undermine its own industry.
Microsoft's TayBot Allegedly Posts Racist, Sexist, and Anti-Semitic Content to Twitter
Microsoft's Tay, an artificially intelligent chatbot, was released on March 23, 2016 and removed within 24 hours due to multiple racist, sexist, and anti-semitic tweets generated by the bot.
Tinder's Personalized Pricing Algorithm Found to Offer Higher Prices for Older Users
Tinder’s personalized pricing was found by Consumers International to consider age as a major determinant of pricing, and could be considered a direct discrimination based on age, according to anti-discrimination law experts.
UK Home Office's Sham Marriage Detection Algorithm Reportedly Flagged Certain Nationalities Disproportionately
UK Home Office's opaque algorithm to detect sham marriages flagged some nationalities for investigation more than others, raising fears surrounding discrimination based on nationality and age.
US DHS’s Opaque Vetting Software Allegedly Relied on Poor-Quality Data and Discriminated against Immigrants
US Citizenship and Immigration Services (USCIS)’s ATLAS software used in vetting immigration requests was condemned by advocacy groups as a threat to naturalized citizens for its secretive algorithmic decision-making, reliance on poor quality data and unknown sources, and alleged discrimination of immigrants using biometric and sensitive information.
Tech Companies Reportedly Influenced Gig Workers' Behaviors Using Algorithms to Vary Pay for Same Amount of Work
Amazon and Uber were alleged in a multiyear ethnographic study using algorithmic systems based on gig workers' data to vary pay, such as by offering them lower wages for the same amount of work.
Coffee Meets Bagel’s Algorithm Reported by Users Disproportionately Showing Them Matches of Their Own Ethnicities Despite Selecting “No Preference”
Users selecting “no preference” were shown by Coffee Meets Bagels’s matching algorithm more potential matches with the same ethnicity, which was acknowledged and justified by its founder as a means to maximize connection rate without sufficient user information.
Major Universities Are Reportedly Using Race as a 'High Impact Predictor' of Student Success
Several major universities are reportedly using a tool that uses race as one factor to predict student success.