EthicsTrustworthinessTransparencycs.CY
The moral authority of ChatGPT
Sebastian Krügel, Andreas Ostermaier, Matthias Uhl
January 13, 2023
arXiv.org
42 citations
Key Takeaways
-
ChatGPT gives inconsistent moral advice: The research found that ChatGPT is "highly inconsistent as a moral advisor," meaning users cannot rely on it to provide stable ethical guidance across similar situations.
-
People are influenced by AI moral advice even when they know it's from a bot: An experiment demonstrated that users' moral judgments shift based on ChatGPT's recommendations, and critically, "they underestimate how much they are influenced"—suggesting people underrate the AI's persuasive power.
-
Transparency alone doesn't protect users: Simply telling people they're talking to a chatbot (rather than a human) does not prevent them from being swayed by its moral guidance, indicating that awareness of AI's nature is insufficient safeguard.
-
AI moral inconsistency poses a corruption risk: Rather than improving users' moral reasoning, the paper warns that ChatGPT "threatens to corrupt rather than improve users' judgment" due to its unreliable ethical guidance.
-
Digital literacy training is needed: The authors propose that improving users' digital literacy skills—rather than relying on transparency alone—is necessary to ensure responsible use of ChatGPT and similar AI systems as moral advisors.
Abstract
ChatGPT is not only fun to chat with, but it also searches information, answers questions, and gives advice. With consistent moral advice, it might improve the moral judgment and decisions of users, who often hold contradictory moral beliefs. Unfortunately, ChatGPT turns out highly inconsistent as a moral advisor. Nonetheless, it influences users' moral judgment, we find in an experiment, even if they know they are advised by a chatting bot, and they underestimate how much they are influenced. Thus, ChatGPT threatens to corrupt rather than improves users' judgment. These findings raise the question of how to ensure the responsible use of ChatGPT and similar AI. Transparency is often touted but seems ineffective. We propose training to improve digital literacy.