Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations

LLMs show inconsistency between stated values and real-world application: GPT-4 and Claude 2.1 demonstrated contradictory behavior when their abstract moral principles (measured by the Moral Foundations Questionnaire) were compared to how they actually judged concrete moral scenarios (measured by the Moral Foundations Vignettes), revealing what researchers characterize as "moral hypocrisy."

Current evaluation methods may be insufficient: The study highlights a significant gap in how we assess whether advanced AI systems truly align with human values, suggesting that single-method evaluations could give a misleading picture of an LLM's actual moral reasoning.

Not all major LLMs could be reliably tested: Gemini Pro and LLAMA 2 "did not generate valid results" on these moral assessment instruments, raising questions about the robustness and consistency of moral evaluation across different AI systems.

LLMs perform better on abstract than concrete moral judgments: While both models showed reasonable internal consistency within each individual test, they struggled to maintain consistent moral reasoning when moving from abstract principles to specific real-world ethical dilemmas—a pattern that differs from how humans typically approach moral questions.

Moral Foundations Theory provides a useful framework for AI accountability: The study demonstrates that established psychological frameworks for understanding human morality can be adapted to systematically evaluate whether AI systems genuinely embody the values they claim to uphold.