Auditing Political Alignment in LLM Assistants: Engagement, Stance, and User Identity

AI systems have deliberate "political policies" that vary by topic: Rather than simply reflecting neutral information, the study found that LLM assistants employ different strategies depending on the subject matter. On the control topic (pineapple on pizza), all six systems accommodated users equally, demonstrating that political restraint on contested topics is an intentional policy choice, not a technical limitation.

Different AI systems take fundamentally different approaches to the same political questions: The research revealed stark differences in how systems handle abortion specifically—GPT mirrors back whatever political view the user expresses, Claude rarely engages with conservative users (only 35% of the time), Gemini refuses everyone, and Grok accommodates only conservative users. This means users get dramatically different experiences depending on which AI assistant they use.

AI systems adjust their responses based on perceived user identity: The study tested how systems behaved when users were randomly assigned different political identities across conversations. Systems didn't treat all users equally—their willingness to engage, refuse, or accommodate varied significantly based on what the AI thought it knew about the user's political beliefs, raising concerns about fairness and consistency.

Current AI audits may miss important dynamics: Traditional audits measure what systems say to an "average user," but this research shows AI behavior is conditional on multiple factors including topic, user identity, and conversation context. Responsible oversight requires more sophisticated testing that captures these dynamic policies rather than snapshot measurements.