Govern, Map, Measure, Absorb: The Legibility Trap in Public Sector Participatory AI Governance

Participation alone doesn't guarantee accountability: The paper analyzes three real public sector AI systems (including predictive policing tools in Detroit and child welfare screening in Allegheny County) and finds that despite documented community engagement, these systems were deployed with minimal changes to their original design, suggesting participation processes may create an illusion of democratic input without substantive influence.

Institutions systematically filter out the most important community concerns: The research identifies a "legibility trap" where government agencies only record and act on community feedback that fits their existing categories and processes, while systematically excluding foundational objections like demands not to deploy the system at all or testimony about structural harms—the very input that would most constrain government power.

Documentation mechanisms themselves can undermine democracy: The paper demonstrates through analysis of official records that the way institutions choose to document community input (what gets written down, how it's categorized, what format it takes) actively shapes which voices are heard and which are ignored, meaning the problem isn't just whether agencies listen, but how they listen.

Ignoring community input causes psychological harm: The research identifies a previously unnamed harm in AI ethics: the repeated experience of communities providing input that is then ignored, which damages people's sense of agency and recognition as subjects whose concerns matter.

Meaningful participation requires structural change, not just procedural compliance: The paper suggests that escaping the legibility trap requires institutions to fundamentally restructure how they document and respond to participation, not simply add more community engagement meetings to existing processes.