Language Models are Alignable Decision-Makers: Dataset and Application to the Medical Triage Domain

Language models can be guided to make decisions reflecting different ethical values: The research demonstrates that large language models like Llama 2, Mistral, and Falcon can be prompted to align their medical triage decisions with specific decision-maker attributes—including ethical principles like fairness and moral desert—without requiring retraining, suggesting a practical pathway for embedding values into AI systems.

Expert decision-makers often disagree, and that's okay: The study acknowledges that in complex scenarios like medical triage, there may be no single "right answer" and different experts legitimately prioritize different values. Rather than forcing AI systems toward one answer, the framework allows them to operate according to different ethical frameworks, which may better reflect real-world decision-making complexity.

AI alignment can be transparent and controllable: By using "zero-shot prompting" (giving instructions without examples), the researchers show that decision-makers can understand and adjust which ethical principles an AI system is following, potentially making AI decision-making more interpretable and trustworthy than black-box approaches.

Open-source tools enable broader accountability: The authors released both their dataset and software publicly, allowing independent researchers and organizations to test, audit, and build upon this work rather than relying on proprietary systems, which supports more democratic oversight of AI development.

Performance improvements are possible through refined techniques: The introduction of "weighted self-consistency" improved the quantified performance of these models, indicating that even with existing AI systems, methodological refinements can enhance reliability in high-stakes decision-making contexts.