Is AI Catching Up to Human Expression? Exploring Emotion, Personality, Authorship, and Linguistic Style in English and Arabic with Six Large Language Models

AI-generated text is currently detectable, but vulnerabilities exist: Machine classifiers can reliably distinguish AI-generated from human-written text with high accuracy (F1>0.95), but this ability breaks down when texts are paraphrased, suggesting detection systems rely on surface-level patterns rather than deeper understanding—a potential security concern as AI becomes more sophisticated.

LLMs express emotion and personality differently than humans: Even when AI generates emotionally or personality-rich content, classifiers trained on human writing perform poorly when analyzing AI text, and vice versa. This indicates that LLMs encode emotional and personality signals in fundamentally different ways than humans do, raising questions about whether AI can truly understand these complex human traits.

Significant gaps exist for under-resourced languages: Arabic, despite being spoken by hundreds of millions, showed particular challenges in personality classification tasks. However, the study found that training systems with AI-generated Arabic text actually improved performance, suggesting synthetic data could help address language inequities—though this requires careful oversight.

Superficial mimicry poses risks for misuse: The fact that AI-generated texts are distinguishable primarily through stylistic cues rather than authentic emotional or personality expression means bad actors could potentially use paraphrasing techniques to evade detection, creating risks for misinformation and impersonation.

Model performance varies significantly: Different LLMs (GPT-4o, Gemini, DeepSeek, etc.) showed different capabilities in mimicking human expression, suggesting that responsible AI deployment requires model-specific evaluation rather than one-size-fits-all approaches.