AI Spam Filters Allegedly Block Legitimate Emails Based on Biased Keyword Detection
Description
AlgorithmWatch tested spam filtering algorithms across Gmail, Yahoo, Outlook, GMX, and LaPoste. Their findings reportedly showed that Microsoft Outlook’s spam filter flagged emails based on specific keywords that led to racial and content-based biases blocking legitimate communications. Emails mentioning Nigeria or containing certain financial and sexual health terms were found to be disproportionately marked as spam.
Actors
Alleged Deployers
- yahoo
- outlook
- laposte
- gmx
- gmail
Alleged Developers
- yahoo
- microsoft
- laposte
- gmx
Harmed Parties
- yahoo!-mail-users
- microsoft-outlook-users
- laposte-users
- gmx-users
- gmail-users
Classification
Risk Domain
Discrimination & ToxicityRisk Subdomain
1.1. Unfair discrimination and misrepresentation
MIT Risk Profile
Topics
Source
Related Incidents (Discrimination & Toxicity)
View all →Research Papers on This Topic
View all →Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Explaining Explanations: An Overview of Interpretability of Machine Learning
RISE: Randomized Input Sampling for Explanation of Black-box Models
Mitigation Tools
Tools related to this risk domain, via shared responsible AI topics.
Open Source
View all →AIF360
Trusted-AI/AIF360
A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.
⭐ 2,872
fairlearn
fairlearn/fairlearn
A Python package to assess and improve fairness of machine learning models.
⭐ 2,199
responsible-ai-toolbox
microsoft/responsible-ai-toolbox
Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.