Outlier Detection Bias Busted: Understanding Sources of Algorithmic Bias through Data-centric Factors

Outlier detection systems—widely used in finance and security—have received far less fairness scrutiny than other AI applications, despite their significant real-world impact. The paper notes that while fairness in supervised machine learning has been extensively studied, unsupervised outlier detection has "attracted little attention" despite numerous high-stakes applications.

Algorithmic bias in outlier detection stems from both data problems AND design choices, not just one or the other. The researchers found that "OD algorithm bias is not merely a data bias problem," meaning that even with perfect data, the algorithms themselves can introduce unfairness through their design.

Outlier detection algorithms show different vulnerabilities to different types of data bias. When researchers injected specific biases into training data—including sample size imbalances, under-representation of groups, measurement noise, and hidden group information—all algorithms studied exhibited fairness problems, but each was susceptible to different types of bias.

Natural differences between groups can create the same fairness problems as intentional bias injection. The paper highlights that data properties emerging from bias injection "could as well be organic," meaning that legitimate differences between groups (like different sparsity levels or variance) can naturally produce the same unfairness patterns as deliberately biased data.