Hybrid Approaches for Moral Value Alignment in AI Agents: a Manifesto

Neither pure rule-based nor pure learning approaches are sufficient alone. The paper identifies that current AI safety methods tend to cluster at two extremes—either hard-coded explicit rules or systems that learn morality entirely from data (like LLMs trained on human feedback)—and argues that hybrid approaches combining both methods are necessary for AI systems that are both controllable and adaptable.

Multiple ethical frameworks should inform AI design. The research examines deontology (rule-based ethics), consequentialism (outcome-focused ethics), and virtue ethics as foundations for AI systems, suggesting that relying on just one ethical framework is insufficient for creating robust moral alignment across diverse real-world scenarios.

Hybrid systems can balance interpretability with flexibility. By combining explicit moral constraints or instructions with learned components, hybrid approaches can maintain human understanding of why an AI system makes decisions (interpretability) while still allowing the system to adapt to new situations—addressing a key tension in current AI safety methods.

Implementation matters as much as theory. The paper demonstrates through case studies using reinforcement learning and language models that the same ethical principles can be implemented in different ways (through intrinsic rewards, moral constraints, or textual instructions), with each approach having distinct practical strengths and weaknesses for real-world deployment.