Agreement Is Not Alignment: Divergent Moral Grounds in Human and LLM Ethical Judgments

Agreement on answers masks disagreement on reasoning: While LLMs often reach the same ethical conclusions as humans, the paper found they rely on fundamentally different moral principles to get there. For example, models and humans might both say "no" to a dilemma, but humans base it on harm prevention while models emphasize justice—making their alignment only superficial.

Current evaluation methods are insufficient: The study demonstrates that testing whether AI systems agree with human judgments on final labels is "misleadingly reassuring" without examining the underlying moral reasoning. This suggests many AI systems currently deemed "aligned" may actually operate on different ethical foundations than intended.

Models systematically prioritize different moral values: Across five domains of ethical judgment, the frontier LLMs tested "redistribute attention" across moral categories like harm, respect, promise-keeping, and justice compared to human annotators. This systematic divergence indicates models aren't simply mimicking human ethics but applying their own moral weighting system.

Responsible AI evaluation requires transparency about reasoning: The paper emphasizes that alignment assessment should include analysis of "reasons, principles, and moral priorities" expressed by models, not just final outputs. This suggests developers need to look deeper into how AI systems justify their decisions to ensure genuine ethical alignment.

The stakes of this gap are significant: Because agreement-based evaluation can be "misleadingly reassuring," organizations deploying AI for high-stakes ethical decisions (healthcare, criminal justice, content moderation) may be overconfident in their systems' true alignment with human values.