AI and Cultural Context: An Empirical Investigation of Large Language Models' Performance on Chinese Social Work Professional Standards

Language models can pass professional exams but lack cultural competence: Seven of eight tested models exceeded the passing score (60 points) on the Chinese National Social Work Examination, yet both Chinese and Western models "showed cultural biases, particularly regarding gender equality and family dynamics," indicating that technical knowledge doesn't translate to culturally appropriate professional reasoning.

Chinese-developed models have regional advantages but aren't universally superior: Chinese models performed better on jurisprudence questions (median 77.0 vs. 70.3), but Western models slightly outperformed on applied knowledge (67.0 vs. 65.5), suggesting that "technical language proficiency does not ensure cultural competence" regardless of model origin.

AI struggles with context-specific professional practice: While models demonstrated "strong professional terminology knowledge," they "struggled with culturally specific interventions," meaning they can recognize correct terms but may not understand how to apply social work principles appropriately within Chinese cultural contexts.

Flawed reasoning can hide behind correct terminology: Between 16.4% to 45.0% of incorrect answers contained "valid reasoning," suggesting models may produce plausible-sounding but ultimately inappropriate guidance—a particular concern for professional fields like social work where incorrect advice affects vulnerable populations.

Deploying AI in non-Western professional contexts requires careful validation: This study demonstrates that AI systems trained primarily on Western data need specific testing and evaluation before deployment in different cultural and professional settings, rather than assuming general competence transfers across contexts.