AI Incidents
Real-world cases where AI systems caused harm, classified by risk domain and entity. Data sourced from 
AIID and classified by 
MIT AI Risk Initiative. CC BY-SA 4.0
197 incidents
Uber's Surge Pricing Reportedly Offered Disproportionate Service Quality along Racial Lines
Uber's surge-pricing algorithm which adjusts prices to influence car availability inadvertently caused better service offering such as shorter wait times for majority white neighborhoods.
Predictive Policing Biases of PredPol
Predictive policing algorithms meant to aid law enforcement by predicting future crime show signs of biased output.
Images of Black People Labeled as Gorillas
Google Photos image processing software mistakenly labelled a black couple as "gorillas."
Google’s YouTube Kids App Presents Inappropriate Content
YouTube’s content filtering and recommendation algorithms exposed children to disturbing and inappropriate videos.
Gender Biases of Google Image Search
Google Image returns results that under-represent women in leadership roles, notably with the first photo of a female "CEO" being a Barbie doll after 11 rows of male CEOs.
UK Visa Streamline Algorithm Allegedly Discriminated Based on Nationality
UK Home Office's algorithm to assess visa application risks explicitly considered nationality, allegedly caused candidates to face more scrutiny and discrimination.
Facebook's Auto-Generated Targeting Ad Categories Contained Anti-Semitic Options
Facebook's automated advertising categories generated using users' declared interests contained anti-Semitic categories such as "Jew hater" and "How to burn Jews" which were listed as fields of study.
Sexist and Racist Google Adsense Advertisements
Advertisements chosen by Google Adsense are reported as producing sexist and racist results.
UT Austin's GRADE Algorithm Reportedly Reduced Review of Lower-Scored PhD Applicants Amid Bias Concerns
From the 2013 through 2019 admissions cycles, UT Austin's Department of Computer Science used GRADE, a statistical machine-learning system trained on past admissions decisions, to score and organize PhD applications. Critics said the system could reproduce historical admissions inequities and reduce attention to lower-scored applicants, while UT Austin said human reviewers still evaluated each file and later discontinued the tool.
Chicago Police's Strategic Subject List Reportedly Biased Along Racial Lines
The Chicago Police Department reportedly used person-based predictive policing models, including the Strategic Subject List and Crime and Victimization Risk Model, to assign violence-risk scores to hundreds of thousands of residents. Official and media reports alleged that the program gave police a mathematically fragile basis for treating heavily policed residents as future victims or suspects, with Black Chicago residents especially exposed to the consequences of that scoring.
Police Reportedly Deployed ShotSpotter Sensors Disproportionately in Neighborhoods of Color
Police departments disproportionately placed ShotSpotter sensors in black and brown neighborhoods, which is denounced by communities for allegedly creating dangerous situations, such as one involving in Adam Toledo's death.
Google Instant's Allegedly 'Anti-Semitic' Results Lead To Lawsuit In France
The organizations SOS Racisme, Union of Jewish Students of France, Movement Against Racism and for Friendship Among Peoples are suing Google due to its autocomplete software suggesting "jewish" when the names of certain public figures were searched on the platform.
IRS Audited Black Taxpayers More Frequently Reportedly Due to Algorithm
The IRS was auditing Black taxpayers more frequently than other groups allegedly due to the design of their algorithms, focusing on easier-to-conduct audits which inadvertently correlated with the group's pattern of tax filing errors.
Automated Adult Content Detection Tools Showed Bias against Women Bodies
Automated content moderation tools to detect sexual explicitness or "raciness" reportedly exhibited bias against women bodies, resulting in suppression of reach despite not breaking platform policies.
Virginia Courts’ Algorithmic Recidivism Risk Assessment Failed to Lower Incarceration Rates
Virginia courts’ use of algorithmic predictions of future offending risks were found by researchers failing to reduce incarceration rates, showed racial and age disparities in risk scores and its application, and neither exacerbated or ameliorated historical racial differences in sentencing.