AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
961 incidents
AI-Designed Phone Cases Are Unexpected
A third-party Amazon merchant named “my_handy_design” was suspected of using a bot to generate cell phone case designs based on the bizarre and unattractive designs being offered.
Atlas Robot Fell off Stage at Conference
Boston Dynamics’s autonomous robot Atlas allegedly caught its foot on a stage light, resulting in a fall off the stage at the Congress of Future Science and Technology Leaders conference.
Airbnb's Trustworthiness Algorithm Allegedly Banned Users without Explanation, and Discriminated against Sex Workers
Airbnb allegedly considered publicly available data on users to gauge their trustworthiness via algorithmic assessment of personality and behavioral traits, resulting in unexplained bans and discriminatory bans against sex workers.
Deepfake Obama Introduction of Deepfakes
University of Washington researchers made a deepfake of Obama, followed by Jordan Peele
Clearview AI Algorithm Built on Photos Scraped from Social Media Profiles without Consent
Face-matching algorithm by Clearview AI was built using scraped images from social media sites such as Instagram and Facebook without user consent, violating social media site policies, and allegedly privacy regulations.
High False Positive Rate by SWP's Facial Recognition Use at Champion's League Final
South Wales Police (SWP)’s automated facial recognition (AFR) at the Champion's League Final football game in Cardiff wrongly identified innocent people as potential matches at an extremely high false positive rate of more than 90%.
BBC Reporter's Twin Brother Cracked HSBC's Voice ID Authentication
HSBC’s voice recognition authentication system was fooled after seven repeated attempts by a BBC reporter's twin brother who mimicked his voice to access his bank account.
Houston ISD's EVAAS Teacher-Evaluation System Reportedly Put Teachers' Jobs at Risk Through Unverifiable Scores
From 2011 to 2015, Houston Independent School District used SAS Institute’s Education Value-Added Assessment System (EVAAS) to rate teacher effectiveness from student test-score growth. Teachers alleged that proprietary, unverifiable EVAAS scores contributed to terminations and contract nonrenewals. On May 4, 2017, a federal judge allowed due-process claims over the system to proceed.
Overfit Kaggle Models Discouraged Data Science Competitors
In the “The Nature Conservancy Fisheries Monitoring” competition on the data science competition website Kaggle, a number of competitors overfit their image classifier models to a poorly representative validation data set.
FaceApp Racial Filters
FaceApp is criticized for offering racist filters.
Facebook Reportedly Outed Sex Workers through Friend Recommendations
Facebook's "People You May Know" feature reportedly outed sex workers by recommending clients to their personal accounts or family members to their business accounts with no option to opt out.
Gender Biases in Google Translate
A Cornell University study in 2016 highlighted Google Translate's pattern of assigning gender to occupations in a way showing an implicit gender bias against women.
Pennsylvania County's Family Screening Tool Allegedly Exhibited Discriminatory Effects
Data analysis by the American Civil Liberty Union (ACLU) on Allegheny County's decision-support Family Screening Tool to predict child abuse or neglect risk found the tool resulting in higher screen-in rates for Black families and higher risk scores for households with disabled residents.
IBM Watson for Oncology Criticized by Customers for Allegedly Unsafe and Inaccurate Cancer Treatment Recommendations
Internal documents from IBM Watson Health showed negative assessments from customers such as Florida’s Jupiter Hospital and Memorial Sloan Kettering criticizing its Watson for Oncology product for allegedly unsafe and incorrect cancer treatment recommendations.
Nissan's "Automatic Emergency Braking" False Positives Posed Traffic Risks to Drivers
Nissan's Automatic Emergency Braking (AEB) feature was reported in a series of complaints for false positives and abrupt braking behaviors, endangering car occupants and traffic participants.