Cross-lingual safety asymmetry in open-weight LLMs

Abstract

Safety alignment in large language models is overwhelmingly trained on English data, yet models are deployed across typologically diverse languages. We evaluate six open-weight large language models (LLMs) across 17 bilingual Korean/Chinese prompt conditions and three translation engines (NLLB-200, Gemini 3.0 Pro, and GPT-5.2), generating 53 040 AdvBench responses. We also conduct French/Spanish and HarmBench robustness checks. The regex classifier is cross-validated against an open-weight LLM judge (Qwen-2.5-72B, $\sim$ 7500 judgments), and we report Wilson 95% confidence intervals. Three findings hold for both classifiers: Safety erosion is language-specific, not generically non-English; Chinese-origin models (Yi, InternLM) exhibit asymmetric vulnerability; and a two-tier pattern persists, where models with target-language safety data resist high-quality translations, and models without such safety data remain vulnerable, regardless of translation quality. These findings reframe cross-lingual safety as a per-language training coverage problem.