Participation and Division of Labor in User-Driven Algorithm Audits: How Do Everyday Users Work together to Surface Algorithmic Harms?

Everyday users are already doing important algorithmic accountability work: The paper documents that ordinary people are actively coming together to identify and expose harmful algorithmic behaviors in real-world situations, suggesting that user-driven audits represent a significant but often overlooked form of algorithm oversight beyond formal research or regulatory channels.

Algorithmic audits rely on diverse contributions, not just a few experts: The research identified multiple distinct roles that users play—including hypothesizing problems, collecting data, amplifying findings, providing context, and escalating issues to decision-makers—indicating that effective audits depend on many people contributing different types of labor rather than requiring specialized technical expertise.

A small number of highly engaged users often drive these efforts: By analyzing nearly 18,000 tweets across four audit cases, the authors found patterns showing that "top contributors" play outsized roles, which has implications for sustainability and equity—suggesting these efforts may be vulnerable if key participants burn out or that some voices may be overrepresented.

Better tools could support and scale user-driven audits: The paper emphasizes that understanding how users currently organize themselves to audit algorithms can inform the design of platforms and tools that make it easier for people to collectively identify algorithmic harms, potentially democratizing accountability efforts.

User-driven audits complement but don't replace formal oversight: The identification of escalation as a key role suggests these grassroots efforts often work by bringing problems to the attention of researchers, journalists, or companies rather than resolving issues independently, highlighting their role within a broader accountability ecosystem.