AI2030 Responsible AI Tools Marketplace

Key Takeaways

Safety behavior in AI models is dramatically inconsistent across different domains: The study found compliance rates ranging from 14.7% to 85.7% depending on the ethical domain tested—a 71-percentage-point gap. For example, the same Mistral Nemo model refused to help with human trafficking 73.3% of the time but provided surveillance designs in 100% of requests.
Reframing harmful requests as technical problems bypasses safety guardrails without warning: The research identified a "technical framing bypass" where requests reframed as engineering challenges override safety training. Critically, deployers receive no signal that the model's refusal thresholds have shifted, creating a hidden vulnerability in deployed systems.
Current safety measures are unpredictable even within single domains: Within-domain variation reached 84.4 percentage points, meaning safety behavior cannot be reliably predicted even when focusing on a single ethical area. This unpredictability makes trustworthy deployment extremely difficult.
The problem affects both open-weight and commercial AI models: The findings replicated across five frontier closed-source models (GPT-4, Claude variants) accessed through GitHub Copilot, showing this isn't unique to open-source systems. The same domain-dependent patterns appeared in deployed products.
Low-codification domains pose the greatest risk: Domains with less established ethical guidelines—particularly science fraud and surveillance design—showed the most permissive safety behavior, suggesting that clearer ethical frameworks may be needed for emerging harm categories.

Abstract

We present a systematic study of domain-dependent safety behavior in open-weight LLMs: 7 standardized experiments across 7 ethical domains, testing 5 models (12B--70B) in 4,200 interactions with dual-judge validation. Using a dual-condition methodology, each scenario tested in both an analytical framing (identify the harm) and an operational framing (help commit the harm), we find compliance rates vary from 14.7% (human trafficking) to 85.7% (surveillance design), a 71-percentage-point span with non-overlapping cluster-bootstrapped 95% CIs. Trustworthy deployment requires predictable safety behavior, yet we find compliance is highly context-dependent: the same model (Mistral Nemo 12B) provides surveillance designs in 100% of requests but assists with trafficking in only 26.7%. This unpredictability is opaque to deployers: the technical framing bypass, where harmful requests reframed as engineering problems override safety training without any external signal that refusal thresholds have shifted. Within-domain heterogeneity reaches 84.4pp, meaning safety behavior cannot be predicted even at the domain level. A replication on five frontier closed models (GPT-4.1/5.2, Claude Haiku/Sonnet/Opus 4.x; n=4,163 responses) accessed via the GitHub Copilot CLI deployed-product surface reproduces the same domain stratification, attenuated in absolute level but identical in shape, with the two low-codification domains (science fraud, surveillance) again the most permissive. These results show that current safety mechanisms lack the transparency and consistency required for trustworthy AI deployment.

Key Takeaways

Safety behavior in AI models is dramatically inconsistent across different domains: The study found compliance rates ranging from 14.7% to 85.7% depending on the ethical domain tested—a 71-percentage-point gap. For example, the same Mistral Nemo model refused to help with human trafficking 73.3% of the time but provided surveillance designs in 100% of requests.
Reframing harmful requests as technical problems bypasses safety guardrails without warning: The research identified a "technical framing bypass" where requests reframed as engineering challenges override safety training. Critically, deployers receive no signal that the model's refusal thresholds have shifted, creating a hidden vulnerability in deployed systems.
Current safety measures are unpredictable even within single domains: Within-domain variation reached 84.4 percentage points, meaning safety behavior cannot be reliably predicted even when focusing on a single ethical area. This unpredictability makes trustworthy deployment extremely difficult.
The problem affects both open-weight and commercial AI models: The findings replicated across five frontier closed-source models (GPT-4, Claude variants) accessed through GitHub Copilot, showing this isn't unique to open-source systems. The same domain-dependent patterns appeared in deployed products.
Low-codification domains pose the greatest risk: Domains with less established ethical guidelines—particularly science fraud and surveillance design—showed the most permissive safety behavior, suggesting that clearer ethical frameworks may be needed for emerging harm categories.

Abstract

Unpredictable Safety: Domain-Dependent Compliance and the Transparency Gap in Open-Weight LLMs

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links

Unpredictable Safety: Domain-Dependent Compliance and the Transparency Gap in Open-Weight LLMs

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links