What should AI see? Using the Public's Opinion to Determine the Perception of an AI

Public opinion significantly shapes AI safety decisions: The study found "highly significant differences" in how different groups (passengers vs. pedestrians, different genders) believe AI should prioritize detecting vulnerable road users, demonstrating that there is no universal consensus on what autonomous vehicles should prioritize seeing.

Who decides AI priorities is an ethical question, not just a technical one: The paper highlights that defining "cost structures" (how much worse it is for AI to misidentify a pedestrian versus a road sign, for example) is fundamentally a values question that requires human input—it cannot be solved by engineers alone.

Perspective matters in AI ethics: Passengers and external traffic participants had different views on detection priorities, suggesting that people's positions relative to AI systems influence their ethical judgments, raising questions about whose values should be centered in AI design.

Current AI design practices may not reflect public values: By surveying the public about detection priorities, the research reveals a gap between how AI systems are typically built (by technical teams) and how the public believes they should work, particularly regarding protection of vulnerable pedestrians.

Participatory approaches can inform responsible AI development: The study demonstrates that online surveys and public input are feasible methods for incorporating diverse perspectives into critical AI decisions, offering a practical pathway for more democratic AI governance in safety-critical applications.