Incident #303·AI Safety & Failures
Google’s Automated Child Abuse Detection Wrongfully Flagged a Parent’s Naked Photo of His Child
Explainability
Description
Google’s automated detection of abusive images of children incorrectly flagged a parent’s photo intended for a healthcare provider, resulting in a false police report of child abuse, and loss of access to his online accounts and information.
Actors
Alleged Deployers
Alleged Developers
Harmed Parties
- a-software-engineer-named-mark
- parents-using-telemedicine-services
Classification
Risk Domain
AI Safety & FailuresRisk Subdomain
7.3. Lack of capability or robustness
MIT Risk Profile
AIUnintentionalPost-deployment
Topics
Explainability
Source
Related Incidents (AI Safety & Failures)
View all →Research Papers on This Topic
View all →Concrete Problems in AI Safety
3,612 citationsJun 2016
A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
2,338 citationsJul 2019
Representation Engineering: A Top-Down Approach to AI Transparency
1,499 citationsOct 2023
Explainable artificial intelligence (XAI) in deep learning-based medical image analysis
1,178 citationsJul 2021
Mitigation Tools
Tools related to this risk domain, via shared responsible AI topics.