SafetyEthicsTrustworthinesscs.HC
Safer Conversational AI as a Source of User Delight
Xiaoding Lu, Aleksey Korshuk, Zongyi Liu, William Beauchamp, Chai Research
April 18, 2023
arXiv.org
3 citations
Key Takeaways
-
Moderation can enhance rather than diminish user satisfaction: The study found that conversational AI systems with appropriate safety measures actually increased user retention, contradicting the common argument that moderation limits enjoyment. This suggests safety and user delight are not inherently at odds.
-
"Heavy-handed" moderation is counterproductive: While moderate safety measures improved user experience, overly restrictive moderation did negatively impact enjoyment. This indicates that the approach to safety matters as much as safety itself—finding the right balance is key.
-
Safety should be thoughtfully designed, not one-size-fits-all: The paper emphasizes the importance of "appropriately defining safety" rather than applying blanket restrictions. This suggests responsible AI requires nuanced decision-making about which safeguards to implement and how strictly to enforce them.
-
User retention is a measurable indicator of responsible AI success: By tracking whether users continued engaging with different AI systems on the Chai platform, the researchers demonstrated that responsible design can be evaluated through concrete user behavior metrics, not just theoretical safety standards.
Abstract
This work explores the impact of moderation on users' enjoyment of conversational AI systems. While recent advancements in Large Language Models (LLMs) have led to highly capable conversational AIs that are increasingly deployed in real-world settings, there is a growing concern over AI safety and the need to moderate systems to encourage safe language and prevent harm. However, some users argue that current approaches to moderation limit the technology, compromise free expression, and limit the value delivered by the technology. This study takes an unbiased stance and shows that moderation does not necessarily detract from user enjoyment. Heavy handed moderation does seem to have a nefarious effect, but models that are moderated to be safer can lead to a better user experience. By deploying various conversational AIs in the Chai platform, the study finds that user retention can increase with a level of moderation and safe system design. These results demonstrate the importance of appropriately defining safety in models in a way that is both responsible and focused on serving users.