FairnessSafetyBiascs.CL
The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It
Zheng-Xin Yong, Beyza Ermis, Marzieh Fadaee, Stephen H. Bach, Julia Kreutzer
May 30, 2025
Conference on Empirical Methods in Natural Language Processing
56 citations
1 influential
Key Takeaways
-
English dominance creates a safety blind spot: The paper's analysis of nearly 300 publications from 2020-2024 reveals that LLM safety research is heavily concentrated on English, meaning safety risks and vulnerabilities in other languages remain largely unmeasured and unaddressed, even for widely-spoken languages.
-
Non-English languages are systematically overlooked: The research shows that non-English languages are rarely studied as standalone subjects in safety research, suggesting that safety improvements developed for English models may not transfer effectively to other languages, potentially leaving non-English speakers exposed to greater risks.
-
Poor documentation practices hinder progress: English-language safety research itself lacks adequate documentation practices, which compounds the problem by making it difficult for researchers to build upon existing work or adapt safety solutions across different languages.
-
The gap is growing, not shrinking: The paper identifies a "significant and growing language gap" in safety research over the 2020-2024 period, indicating that despite increased attention to AI safety overall, the field is becoming more rather than less English-centric.
-
Concrete solutions exist but require coordinated effort: The authors propose three specific research directions—safety evaluation methods, training data generation, and cross-lingual safety generalization—that could help develop more inclusive AI safety practices for the world's diverse populations.