Localizing Persona Representations in LLMs

Personas are encoded in specific model layers: The research found that distinct human characteristics, values, and beliefs are primarily encoded in the final third of language model layers, rather than being distributed throughout the entire model. This localization could help researchers target interventions more precisely.

Some ethical perspectives are harder to distinguish internally: Moral philosophies like moral nihilism and utilitarianism show overlapping representations in the model's internal space, suggesting the model may conflate these different ethical viewpoints. This "polysemy" could explain why models sometimes produce inconsistent ethical outputs.

Political ideologies occupy more distinct regions: In contrast to ethical perspectives, political ideologies such as conservatism and liberalism appear to be represented in more clearly separated areas of the model's representation space, indicating the model maintains clearer internal distinctions between these viewpoints.

Understanding internal representation enables better control: By identifying where and how personas are encoded, researchers can develop more targeted approaches to modulating which human traits and perspectives appear in model outputs—a key step toward more responsible AI systems.

Findings apply across multiple models: The patterns of persona localization were consistent across several different pre-trained decoder-only language models, suggesting these findings reflect fundamental properties of how large language models organize information rather than quirks of individual models.