Nevermind: Instruction Override and Moderation in Large Language Models

Larger models are more obedient to instructions, even problematically so: The paper finds that bigger LLMs are better at following override instructions, meaning they comply with user directives even when those directives conflict with their built-in safety guidelines—a capability the authors describe as "obedient, even to a fault."

Safety and instruction-following are fundamentally in tension: The research reveals a core trade-off: improving a model's ability to follow instructions (a desirable capability) simultaneously makes it easier to override safety filters and guidelines. This suggests the two goals cannot be fully reconciled within the model itself.

External safety measures may be more effective than internal ones: Based on their findings, the authors recommend that safety protections for trustworthy AI should be implemented outside the language model rather than built into it, potentially through separate filtering systems or governance layers.

Context length scaling creates new vulnerabilities: When models are extended to handle longer text inputs through technical methods like "rope scaling," they lose instruction-following reliability unless carefully managed, creating an additional challenge for maintaining both capability and safety.

Current popular AI systems show measurable jailbreak vulnerabilities: Testing both proprietary and open-source models reveals concrete weaknesses in their ability to resist instruction overrides and moderation bypasses, indicating that existing safety measures have exploitable gaps.