WEIRD Audits? Research Trends, Linguistic and Geographical Disparities in the Algorithm Audits of Online Platforms -- A Systematic Literature Review

Algorithm auditing research is heavily concentrated on a few major platforms: The systematic review of 176 studies found significant skew toward examining only a handful of online platforms, meaning we have much less understanding of how algorithms work on many other widely-used services.

Western bias limits our understanding of global algorithmic harms: Research overwhelmingly focuses on Western contexts, particularly the United States, and uses English-language data, which means algorithmic discrimination affecting non-Western populations and non-English speakers remains largely unstudied and undocumented.

Researchers oversimplify how they measure bias and discrimination: The studies tend to focus on a narrow set of demographic categories (like race or gender) and operationalize them in overly simplified ways, potentially missing more complex and nuanced forms of algorithmic bias that don't fit neatly into these categories.

Geographic disparities in who conducts auditing research matter: The authors found significant linguistic and geographical disparities in where auditing research originates, suggesting that perspectives from researchers outside Western institutions are underrepresented in shaping how we understand and evaluate algorithmic systems.

Current auditing landscape leaves critical gaps in coverage: The combination of platform concentration, geographic bias, and simplified measurement approaches means there are substantial blind spots in our knowledge about algorithmic fairness—particularly for populations and contexts outside the Western focus of existing research.