BIASEDTALES-ML: A Multilingual Dataset for Analyzing Narrative Attribute Distributions in LLM-Generated Stories

AI safety testing is heavily English-biased: The paper reveals that evaluations of LLM alignment and safety—including how models handle bias—focus primarily on English, leaving a significant gap in understanding how these safeguards work across different languages. This means we may have a false sense of security about AI safety in non-English contexts.

Bias in AI stories varies dramatically by language: The researchers found "substantial cross-lingual variability in narrative generation patterns," meaning that biases present in English-generated children's stories don't necessarily appear the same way in other languages. This suggests that bias mitigation efforts tuned for English may not transfer effectively to other linguistic communities.

Lower-resource languages face greater risks: The study specifically notes that "distributions observed in English do not always exhibit similar characteristics in other languages, particularly in lower-resource settings." This raises concerns that children in communities with less-resourced languages may encounter different—and potentially more problematic—biases in AI-generated educational content.

Narrative patterns reveal hidden biases: Beyond surface-level language differences, the analysis identified "recurring structural patterns involving character roles, settings, and thematic emphasis" that vary across languages. This suggests biases in AI storytelling are embedded in deeper narrative structures, not just word choice, making them harder to detect and fix.

Current AI evaluation methods are insufficient for global deployment: The paper demonstrates that English-centric testing cannot adequately characterize how AI systems behave in multilingual, multicultural contexts—a critical limitation given that LLMs are being deployed worldwide for content generation affecting diverse populations.