Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms

Current AI auditing processes exclude marginalized communities: Traditional bias evaluation benchmarks and model documentation fail to integrate knowledge from the communities most affected by AI harms, creating a power imbalance between auditors and impacted groups.

Bias bounties alone are insufficient: While bias bounties represent progress in engaging communities to identify AI harms, the paper's workshop with queer communities revealed that participants wanted to go "far beyond what bias bounties afford," questioning the ownership structures, financial incentives, and actual effectiveness of these programs.

Marginalized communities have been rarely asked what they want: The research highlights a critical gap—despite proposing community-engaged auditing processes, researchers and practitioners have neglected to ask marginalized communities (in this case, queer communities) what they actually desire from these evaluation processes.

Community ownership should replace extractive models: Rather than treating marginalized communities as sources of feedback within existing bounty structures, the authors advocate for communities to have ownership over auditing processes themselves, shifting power dynamics away from external auditors.

Participatory co-creation should complement technical auditing: The paper recommends combining bias bounties with deeper participatory processes like co-creation, suggesting that responsible AI evaluation requires ongoing collaboration rather than one-time feedback collection.