FairnessBiasSecuritycs.LG
Exacerbating Algorithmic Bias through Fairness Attacks
Ninareh Mehrabi, Muhammad Naveed, Fred Morstatter, Aram Galstyan
December 16, 2020
AAAI Conference on Artificial Intelligence
85 citations
9 influential
Key Takeaways
-
Fairness measures in AI systems can be deliberately undermined through data poisoning attacks, meaning that even algorithms designed to be fair can be manipulated by adversaries who inject malicious data into training datasets—a vulnerability that has received far less attention than attacks on accuracy.
-
Two specific attack strategies can effectively compromise fairness: "anchoring attacks" that skew decision boundaries by strategically placing false data points, and "influence attacks" that artificially increase the correlation between sensitive attributes (like race or gender) and algorithmic outcomes, both of which the researchers demonstrated to be effective in their experiments.
-
Current fairness safeguards may be insufficient, as the paper reveals that most adversarial machine learning research has focused exclusively on protecting accuracy rather than fairness, leaving fairness measures as a largely undefended target for malicious actors.
-
Organizations deploying "fair" AI systems need to consider data security as part of fairness protection, since the robustness of fairness measures depends not just on algorithm design but also on preventing tampering with training data—suggesting that responsible AI requires both technical fairness measures and strong data governance practices.
Abstract
Algorithmic fairness has attracted significant attention in recent years, with many quantitative measures suggested for characterizing the fairness of different machine learning algorithms. Despite this interest, the robustness of those fairness measures with respect to an intentional adversarial attack has not been properly addressed. Indeed, most adversarial machine learning has focused on the impact of malicious attacks on the accuracy of the system, without any regard to the system's fairness. We propose new types of data poisoning attacks where an adversary intentionally targets the fairness of a system. Specifically, we propose two families of attacks that target fairness measures. In the anchoring attack, we skew the decision boundary by placing poisoned points near specific target points to bias the outcome. In the influence attack on fairness, we aim to maximize the covariance between the sensitive attributes and the decision outcome and affect the fairness of the model. We conduct extensive experiments that indicate the effectiveness of our proposed attacks.