Appearing Legitimate is Not Enough: Interrogating Synthetic Agents in Representational Processes through a Participatory Design Lens

Synthetic agents cannot legitimately replace human participation in democratic processes. The paper argues that appearing realistic or producing useful information is insufficient; actual human participation is a necessary condition for the legitimacy of democratic institutions like policy consultation, jury deliberation, and diplomatic processes. Legitimacy and personhood are "mutually constitutive," meaning you cannot have one without the other.

The risks scale with institutional importance. The researchers examined three case studies across different scales—local policy, enterprise jury deliberation, and global diplomacy—demonstrating that using synthetic agents as human substitutes becomes increasingly problematic as the stakes and scope of representation grow, particularly in contexts where democratic legitimacy depends on genuine human voices.

"Appearing legitimate" creates a false sense of validity. The paper's title emphasizes a critical concern: synthetic agents can be designed to seem credible and produce plausible outputs, but this surface-level legitimacy masks fundamental representational and ethical problems. This distinction matters because decision-makers might be misled into treating synthetic participation as equivalent to human participation.

Participatory Design offers a framework for identifying representational risks. The authors propose using Participatory Design's engagement modes (probing, priming, understanding, and generating) as tools to interrogate whether synthetic agents are appropriate for specific contexts, helping organizations think critically about when and where such substitution is ethically problematic.

Current uses of synthetic agents in research and policy need ethical boundaries. The paper identifies both "soft and hard boundaries" for designing with synthetic agents, suggesting that while some applications may be acceptable, others—particularly those involving democratic representation—should be restricted or prohibited entirely.