From a River in Gilead to the Inference Distributions of Large Language Models: Covert Dialect Bias and Linguistic Profiling at Scale

Hidden bias persists despite safety measures: While alignment techniques successfully reduce explicit racial bias in LLM outputs, they fail to address covert attitudinal associations embedded in the models' internal probability distributions—meaning the bias remains detectable through technical analysis even when not visible in generated text.

Dialect discrimination operates at scale across multiple models: All ten open-weight LLMs tested showed consistent patterns of penalizing African American Vernacular English (AAVE) and Nigerian Pidgin (NP) by associating them with more negative adjectives than Standard American English, suggesting this is a systemic issue rather than isolated to specific models.

Different dialects face distinct stereotype clusters: Rather than treating all non-standard dialects equally, LLMs apply different negative stereotypes to different varieties—for example, AAVE and NP each trigger their own specific stereotype patterns—indicating the models have absorbed nuanced but harmful cultural associations.

Social context amplifies bias in high-stakes decisions: The bias intensifies as social proximity increases (from formal tenant screening to roommate selection), which is particularly concerning since LLMs are increasingly deployed in housing and hiring decisions where such discrimination has real consequences for people's lives.

Prestige and postcolonial context matter: Nigerian Standard English, which carries institutional prestige, was initially favored in formal contexts but increasingly penalized as situations became more personal, revealing that LLMs encode complex hierarchies around language, race, and geography that shift based on context.