SafetyAccountabilityPrivacycs.CL
Safety Compliance: Rethinking LLM Safety Reasoning through the Lens of Compliance
Wenbin Hu, Huihao Jing, Haochen Shi, Haoran Li, Yangqiu Song
September 26, 2025
arXiv.org
5 citations
1 influential
Key Takeaways
-
Legal frameworks can provide systematic safety standards: Rather than relying on ad-hoc safety rules, the paper proposes using established legal standards like the EU AI Act and GDPR as rigorous benchmarks for LLM safety, offering a more comprehensive and systematic approach to defining what "safe" means for AI systems.
-
Current safety methods are insufficient for complex AI behaviors: The research identifies that existing LLM safety approaches lack systematic protection and fail to address the nuanced, complex behaviors of modern language models, suggesting that piecemeal safety measures are inadequate for today's advanced AI systems.
-
Safety compliance can be measured and improved: The authors developed a new benchmark based on legal statutes and trained a "Compliance Reasoner" that improved performance by over 10% on both EU AI Act and GDPR compliance metrics, demonstrating that legal alignment is both measurable and achievable through targeted training.
-
Bridging law and AI safety is practical: By grounding LLM safety in real legal requirements rather than abstract principles, this approach creates a concrete framework that regulators, developers, and users can understand and verify, making AI safety more transparent and accountable.
Abstract
The proliferation of Large Language Models (LLMs) has demonstrated remarkable capabilities, elevating the critical importance of LLM safety. However, existing safety methods rely on ad-hoc taxonomy and lack a rigorous, systematic protection, failing to ensure safety for the nuanced and complex behaviors of modern LLM systems. To address this problem, we solve LLM safety from legal compliance perspectives, named safety compliance. In this work, we posit relevant established legal frameworks as safety standards for defining and measuring safety compliance, including the EU AI Act and GDPR, which serve as core legal frameworks for AI safety and data security in Europe. To bridge the gap between LLM safety and legal compliance, we first develop a new benchmark for safety compliance by generating realistic LLM safety scenarios seeded with legal statutes. Subsequently, we align Qwen3-8B using Group Policy Optimization (GRPO) to construct a safety reasoner, Compliance Reasoner, which effectively aligns LLMs with legal standards to mitigate safety risks. Our comprehensive experiments demonstrate that the Compliance Reasoner achieves superior performance on the new benchmark, with average improvements of +10.45% for the EU AI Act and +11.85% for GDPR.