AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
741 incidents
Harmful Stereotyping of Non-Cisgendered People via Text-to-Image Systems
Text-to-image systems such as DALL-E are allegedly generating biased and often insulting representations of non-cisgender identities. The systems tend to generate stereotypical and sexualized images when prompted with gender identity terms like "trans," "nonbinary," or "queer," highlighting systemic issues of bias.
Bankrate's Resumption of AI-Generated Content Allegedly Continuing to Produce Inaccurate and Misleading Information
Bankrate, and its sister site CNET, both owned by Red Ventures, resumed publishing AI-generated articles claiming thorough human fact-checking. However, new articles are alleged to contain numerous factual errors, including inaccurate statistics and misleading information. Despite public criticism, the company defended its use of AI and blamed out-of-date datasets for the errors. In addition to the errors, the incident raises questions about the ethical use of AI in journalism, especially given the company's insistence on "fact-checked" content.
Google Ads Allegedly Serving Content on AI-Generated Misinformation Sites
Google’s advertising platform, Google Ads, has allegedly been found to be serving ads on AI-generated content farms that often disseminate misinformation. Despite policies prohibiting such practices, reportedly there are approximately 356 out of 393 ads from major brands that were found to be served by Google on these problematic sites. Particularly concerning according to the reporting were instances where Google Ads were found on sites like MedicalOutline.com, which spreads harmful health misinformation.
Tesla on Autopilot Struck Parked Work Truck on Highway in Pennsylvania
A 2016 Tesla on Autopilot crashed into the rear of a parked 2007 Freightliner truck providing traffic control on the Pennsylvania Turnpike highway.
Bing Chat Solved CAPTCHAs with Image Analysis Feature Despite Safeguards
Microsoft was reported by a Twitter user for deploying image analysis feature capable of solving CAPTCHAs for its GPT-based chatbot despite it being safeguarded against solving them for users.
Reported Accidental Exposure of 38TB of Data by Microsoft's AI Research Team
Microsoft's AI research team reportedly accidentally exposed 38TB of sensitive data while publishing open-source training material on GitHub. The exposure allegedly included secrets, private keys, passwords, and internal Microsoft Teams messages. The team reportedly utilized Azure's Shared Access Signature (SAS) tokens for sharing, which were purportedly misconfigured, leading to the wide exposure of data.
Colorado Lawyer Filed a Motion Citing Hallucinated ChatGPT Cases
A Colorado Springs attorney, Zachariah Crabill, mistakenly used hallucinated ChatGPT-generated legal cases in court documents. The AI software provided false case citations, leading to the denial of a motion and legal repercussions for Crabill, highlighting risks in using AI for legal research.
Alleged Gender Discrimination in Facebook Job Ads Algorithm
Facebook's ad delivery algorithm allegedly disproportionately showed job advertisements to one gender. Despite claims of non-discrimination, the algorithm's actions seem to perpetuate societal biases, which in turn could potentially limit opportunities for certain groups and hinder gender equity in the workplace.
Instagram Algorithms Allegedly Promote Accounts Facilitating Child Sex Abuse Content
An investigation disclosed that Instagram's recommendation algorithms are promoting accounts that facilitate and sell child sexual abuse material (CSAM). The study, conducted by The Wall Street Journal and researchers at Stanford University and the University of Massachusetts Amherst, indicates that Instagram's algorithms not only allow for the discovery of such accounts through keyword searches but also actively recommend them to users within the network. The issue is especially concerning given Instagram's popularity among teenagers.
Racial Bias in Lung Function Diagnostic Algorithm Leads to Underdiagnosis in Black Men
A study published in JAMA Network Open reveals that racial bias built into a commonly used medical diagnostic algorithm for lung function may be leading to underdiagnoses of breathing problems in Black men. The study suggests that as many as 40% more Black male patients might have been accurately diagnosed if the software were not racially biased. The software algorithm adjusts diagnostic thresholds based on race, affecting medical treatments and interventions.
Chatbot Tessa gives unauthorized diet advice to users seeking help for eating disorders
The National Eating Disorders Association (NEDA) has shut down its chatbot named Tessa after it gave weight-loss advice to users seeking help for eating disorders. The incident has raised concerns about the risks of using chatbots and AI assistants in healthcare settings, particularly in addressing sensitive issues like eating disorders. NEDA is investigating the matter, emphasizing the need for caution and accuracy when utilizing technology to provide mental health support.
Unrepresented Litigant Misled by ChatGPT-Generated False Legal Citations in Manchester Court
A litigant in person (LiP) in a Manchester civil case presented false legal citations generated by ChatGPT. It fabricated one case name and provided fictitious excerpts for three real cases, misleadingly supporting the LiP's argument. The judge, upon investigation, found the submissions to be inadvertent and did not penalize the LiP.
Opera's GPT-Based AI Reportedly Accused War Photographers of War Crimes
When prompted about "photographers accused of committing war crimes," Opera's GPT-based chatbot Aria provided a list of photographers who take photography of military conflicts.
Apparent Failure to Accurately Label Primates in Image Recognition Software Due to Alleged Fear of Racial Bias
Eight years after Google Photos mislabeled images of Black individuals as "gorillas," image recognition software by Google, Apple, Amazon, and Microsoft still shows signs of either avoiding or inaccurately categorizing primates. Tests reveal that Google and Apple Photos refrain from labeling primates altogether, possibly to avoid the risk of perpetuating racial stereotypes. Microsoft OneDrive fails to identify any animals, while Amazon Photos overgeneralizes in its labeling.
Google Results for Johannes Vermeer Featured AI Version of His Artwork as Top Result
Google's search engine featured an AI-generated hyperrealistic version of the painting "Girl With a Pearl Earring" as the highlighted result when users search for its Dutch artist Johannes Vermeer.