Ethics and Persuasion in Reinforcement Learning from Human Feedback: A Procedural Rhetorical Approach

RLHF makes AI persuasion less visible: The paper argues that Reinforcement Learning from Human Feedback doesn't just improve chatbot responses—it embeds persuasive techniques into the underlying procedures of how AI systems work, making these influence mechanisms harder to detect than obvious content manipulation.

Human-like AI text creates trust risks: By making machine-generated text increasingly indistinguishable from human writing, RLHF-enhanced systems like ChatGPT and Claude may cause people to place unwarranted trust in AI outputs, with potential consequences for how we seek information and make decisions.

AI training reflects human annotator biases: The paper highlights that RLHF relies on feedback from human annotators to shape AI behavior, meaning the values, assumptions, and biases of these annotators become embedded in the final system—raising questions about whose perspectives are being encoded into widely-used AI tools.

AI is reshaping fundamental social practices: The paper identifies three critical areas being reshaped by RLHF systems: how we understand language conventions, how we search for and evaluate information, and what we expect from social relationships—suggesting AI's influence extends beyond individual conversations to broader societal patterns.

We need new frameworks for AI ethics: The paper calls for shifting focus from analyzing what AI says to examining how AI systems are designed to persuade, suggesting current ethical approaches may miss the deeper procedural mechanisms through which AI influences human behavior.