AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
244 incidents
AI-Assisted Body Scanners Reportedly Subjected Transgender Travelers to Invasive Body Searches
Transportation Security Administration (TSA)'s use of image-processing body scanners at airports led transgender and gender-nonconforming travelers to be subjected to allegedly discriminatory and invasive searches, such as being asked to remove undergarments in private rooms by officers not of their gender.
"Jewish Baby Strollers" Provided Anti-Semitic Google Images, Allegedly Resulting from Hate Speech Campaign
Google's Image search for "Jewish baby strollers" showed offensive, anti-Semitic results, allegedly a result of a coordinated hate-speech campaign involving malicious actors on 4chan.
AI-Designed Phone Cases Are Unexpected
A third-party Amazon merchant named “my_handy_design” was suspected of using a bot to generate cell phone case designs based on the bizarre and unattractive designs being offered.
Airbnb's Trustworthiness Algorithm Allegedly Banned Users without Explanation, and Discriminated against Sex Workers
Airbnb allegedly considered publicly available data on users to gauge their trustworthiness via algorithmic assessment of personality and behavioral traits, resulting in unexplained bans and discriminatory bans against sex workers.
FaceApp Racial Filters
FaceApp is criticized for offering racist filters.
Gender Biases in Google Translate
A Cornell University study in 2016 highlighted Google Translate's pattern of assigning gender to occupations in a way showing an implicit gender bias against women.
Pennsylvania County's Family Screening Tool Allegedly Exhibited Discriminatory Effects
Data analysis by the American Civil Liberty Union (ACLU) on Allegheny County's decision-support Family Screening Tool to predict child abuse or neglect risk found the tool resulting in higher screen-in rates for Black families and higher risk scores for households with disabled residents.
Social Media's Automated Word-Flagging without Context Shifted Content Creators' Language Use
TikTok's, YouTube's, Instagram's, and Twitch's use of algorithms to flag certain words devoid of context changed content creators' use of everyday language or discussion about certain topics in fear of their content getting flagged or auto-demonetized by mistake.
Uber Locked Indian Drivers out of Accounts Allegedly Due to Facial Recognition Fails
Uber drivers in India reported being locked out of their accounts allegedly due to Real-Time ID Check's facial recognition failing to recognize appearance changes or faces in low lighting conditions.
High-Toxicity Assessed on Text Involving Women and Minority Groups
Google's Perspective API, which assigns a toxicity score to online text, seems to award higher toxicity scores to content involving non-white, male, Christian, heterosexual phrases.
Alexa Plays Pornography Instead of Kids Song
An Amazon Echo Dot using the Amazon Alex software started to play pornographic results when a child asked it to play a song.
Passport checker Detects Asian man's Eyes as Closed
New Zealand passport robot reader rejects the application of an applicant with Asian descent and says his eyes are closed.
NYPD's Deployment of Facial Recognition Cameras Reportedly Reinforced Biased Policing
New York Police Department’s use of facial recognition deployment of surveillance cameras were shown using crowdsourced volunteer data reinforcing discriminatory policing against minority communities.
LinkedIn Search Prefers Male Names
An investigation by The Seattle Times in 2016 found a gender bias in LinkedIn's search engine.
AI Beauty Judge Did Not Like Dark Skin
In 2016, after artificial inntelligence software Beauty.AI judged an international beauty contest and declared a majority of winners to be white, researchers found that Beauty.AI was racially biased in determining beauty.