Personality Without Persons? A Psychometric Critique of Big Five Testing in Large Language Models

Big Five personality tests are not valid for LLMs: While the original human-developed Big Five inventory items fail to appropriately describe LLMs, even adapted versions only achieve "acceptable" content validity, suggesting these human-designed tools fundamentally may not measure equivalent constructs in AI systems.

Personality scores don't meaningfully distinguish between different LLMs: Between-model variance accounts for only 7-17% of total score variance, meaning Big Five tests fail to capture meaningful differences across the 264 LLMs tested—raising questions about their utility for comparing AI systems.

LLMs don't exhibit the same personality structure as humans: Four of the five personality dimensions collapse into one factor (with correlations ≥ .90), meaning LLMs don't reproduce the canonical five-factor structure of human personality, indicating the underlying construct is fundamentally different.

Alignment training artificially inflates "desirable" personality traits: Instruction-tuned model variants show shifts toward socially desirable Big Five profiles compared to base models, suggesting personality scores may reflect training objectives rather than genuine model characteristics—a critical concern for using these metrics in governance decisions.

Current personality assessments should not inform AI governance: The paper's findings demonstrate that Big Five inventories do not measure constructs equivalent to human personality in LLMs, undermining their credibility as a basis for comparing systems or making downstream policy claims about AI behavior and safety.