Can Valence Reflect Morality in Natural Language? A Preliminary Annotation Study

Feelings matter for AI ethics: Current AI ethics implementations largely ignore affect (emotions/feelings), yet major ethical theories—including Virtue Ethics, Consequentialism, and Deontological Ethics—all recognize that human feelings play a role in moral judgments and actions. This suggests AI systems designed to align with human ethics should similarly account for emotional dimensions.

Valence (emotional tone) can predict morality in text: The study created a dataset of 500 annotations measuring "moral valence" (ranging from -1 to 1) for scenarios, and found that these emotional tone measurements showed "significant relationships" with moral classifications. Using regularized logistic regression, the valence features achieved a Matthew's correlation coefficient of 0.764 for binary moral/immoral classification, providing "early evidence of the usefulness of valence features for morality estimation."

Both actions and their consequences carry emotional weight: The research distinguished between the valence of actions/judgments themselves and the valence of their consequences for others, suggesting that responsible AI systems may need to evaluate both the emotional character of decisions and their emotional impact on affected parties.

Moral Foundations Theory supports emotion-centered approaches: The study notes that Moral Foundations Theory—a prominent descriptive ethics framework—positions feelings as "central to many of its foundations," providing theoretical support for incorporating affect into AI ethics implementations.