AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
961 incidents
Dartmouth's Geisel School of Medicine Reportedly Used Canvas Activity Logs to Accuse Students of Cheating
Dartmouth's Geisel School of Medicine reportedly used Canvas learning-management activity logs and an internal analysis process to investigate remote-exam cheating during the 2020–2021 academic year. Seventeen medical students were charged after the school inferred that they accessed course materials during exams, but students and outside technical reviewers said automated Canvas activity may have been misread as intentional misconduct. Dartmouth later dropped the charges.
Year-long AI Surveillance Pilot in Two South Australian Aged Care Facilities Reportedly Overwhelmed Staff with False Positives
Between March 2021 and March 2022, an AI-enabled video and audio monitoring system was trialed in two South Australian aged care facilities. According to an independent audit commissioned by South Australia Health, the system produced over 12,000 false alerts, overwhelming staff and contributing to at least one missed real incident. The report concluded the system did not reach accuracy levels acceptable to staff or management during the pilot.
Facebook's Automated Tools Failed to Adequately Remove Hate Speech, Violence, and Incitement
Facebook's automated moderation tools were shown by internal documents performing incomparably to human moderators, and accounting for only a small fraction of hate speech, violence, and incitement content removal.
TikTok’s "For You" Allegedly Pushed Fatal “Blackout” Challenge Videos to Two Young Girls
TikTok’s recommendation algorithm was alleged in a lawsuit to have intentionally and repeatedly pushed videos of the “blackout” challenge onto children’s feeds, incentivizing their participation which ultimately resulted in the death of two young girls.
Manufacturing Robot Failure Caused Factory Worker's Death in India
A sensor snag resulted in an automotive parts factory robot falling on a factory worker in India
Facebook’s and Twitter's Automated Content Moderation Reportedly Failed to Effectively Enforce Violation Rules for Small Language Groups
Facebook's and Twitter were not able to sufficiently moderate content of small language groups such as the Balkan languages using AI, allegedly due to the lack of investment in human moderation and difficulty in AI-solution design for the languages.
California's Algorithm Considered ZIP Codes in Vaccine Distribution, Allegedly Excluding Low-Income Neighborhoods and Communities of Color
California's vaccine-distribution algorithm used ZIP codes as opposed to census tracts in its decision-making, which critics said undermined equity and access for vulnerable communities who are largely low-income, underserved neighborhoods with low Healthy Places Index scores.
Facebook's Advertisement Moderation System Allegedly Routinely Misidentified Adaptive Fashion Products as Medical Equipment and Blocked Their Sellers
Facebook platforms' automated ad moderation system falsely classified adaptive fashion products as medical and health care products and services, resulting in regular bans and appeals faced by their retailers.
AI-Generated Deepfakes Reportedly Derailed Political Career of Florida Official
Sabrina Javellana, a Florida politician, was reportedly targeted with AI-generated deepfake pornography in February 2021, which was spread online, leading to severe emotional distress and her eventual withdrawal from public life.
Remote Proctoring Facial-Detection Software Reportedly Failed to Detect Black Student During Lab Quiz
In February 2021, a Black college student, Amaya Ross, reportedly tried to take a remote biology lab quiz using proctoring software that could not detect her face. Ross said she spent 45 minutes changing lights, shades, position, and background before pointing an LED flashlight at her face so the app would work; the quiz itself was only 30 minutes.
Amazon’s Search and Recommendation Algorithms Found by Auditors to Have Boosted Products That Contained Vaccine Misinformation
Evidence of the "filter-bubble effect" were found by vaccine-misinformation researchers in Amazon's recommendations, where its algorithms presented users who performed actions on misinformative products with more misinfomative products.
Facebook's Automated Moderation Mistakenly Flagged Landmark's Name as Offensive
Facebook's automated system mistakenly labelled posts featuring the seafaring landmark Plymouth Hoe as misogynistic.
Facebook Recommended Military Gear Ads Despite Pause on Weapons Accessories Ads
Facebook’s algorithmic recommendations reportedly continued showing advertisements for gun accessories and military gear, despite Facebook’s halt on weapons accessories ads following the US Capitol attack.
Israeli Tax Authority Reportedly Used an Opaque Automated System to Issue a Fine, Declining to Explain or Disclose the Underlying Calculation
An Israeli farmer, Moshe Har Shemesh, reportedly received a fine generated by a Tax Authority software system whose calculation officials were allegedly unable to explain. When the farmer reportedly sought access to the program or its source code to understand the basis for the amount, the authority allegedly refused, citing security concerns and the difficulty of extracting the embedded guidelines. The dispute reportedly later moved into legal proceedings focused on whether code and automated decision rules constitute information subject to public disclosure.
XPeng Motors Fined For Illegal Collection of Consumers’ Faces Using Facial Recognition Cameras
The Chinese electric vehicle (EV) firm XPeng Motors was fined by local market regulators for illegally collecting in-store customers’ facial images without their consent for six months.