Persuasion and Safety in the Era of Generative AI

LLMs are becoming sophisticated persuaders: Large language models have developed advanced capabilities to persuade people, raising urgent safety concerns that regulators like the EU are already addressing through legislation prohibiting manipulative AI systems.

Not all persuasion is harmful: The research distinguishes between "rational persuasion" (which appeals to reason) and "manipulation" (which exploits cognitive biases), suggesting that ethical persuasion is possible—the key is understanding the difference and preventing the latter.

We lack tools to identify manipulation: The paper highlights a critical gap: there are few empirical studies or resources available to detect when AI systems are using manipulative versus legitimate persuasive techniques, making it difficult to enforce responsible AI practices.

Datasets and taxonomies are essential safeguards: By creating a human-annotated dataset and taxonomy of persuasive techniques, this work provides concrete resources that can help developers and regulators identify and mitigate risks from persuasive AI systems.

Ethical persuasion requires ongoing discussion: The research calls for broader conversations about what constitutes acceptable persuasion in an age of generative AI, suggesting this is not just a technical problem but a societal one requiring input from multiple stakeholders.