Language Models Should be Used to Surface the Unwritten Code of Science and Society

LLMs can reveal society's "unwritten rules": The paper demonstrates that language models can be used to surface implicit biases and unstated expectations that guide human decision-making—in this case, showing that peer reviewers reward papers based on factors they rarely explicitly mention in their reviews.

Peer review has hidden criteria beyond stated standards: The research found that academic reviewers systematically favor papers with strong "storytelling" and positioning within existing literature, yet rarely articulate this preference in their written comments. This gap between implicit rewards (reflected in scores) and explicit feedback (in review text) suggests reviewers may not be fully aware of their own evaluation criteria.

There's a disconnect between what reviewers say and what they reward: The study found a correlation of 0.49 between reviewer scores and LLM-identified factors aligned with stated scientific norms (like theoretical rigor), but a negative correlation of -0.14 for contextualization factors—meaning reviewers penalize explicit mention of these criteria while rewarding them in practice.

Making bias visible enables critique and accountability: By using LLMs to articulate the hidden heuristics embedded in their training data, the framework creates an opportunity for the scientific community to examine and debate whether these implicit standards are fair, appropriate, and aligned with stated values.

This approach could extend beyond academia: The methodology could potentially be applied to other domains where unwritten rules influence decisions—hiring, lending, content moderation—making systemic biases more transparent and subject to public scrutiny.