BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models

Backdoor attacks on AI safety systems are a real threat: The paper demonstrates that attackers can embed hidden "triggers" in language models that cause them to generate harmful content (like malicious code) while appearing safe during normal use, with attack success rates exceeding 95% before defense.

A new defense method called BEEAR can effectively neutralize these attacks: The approach reduced backdoor attack success rates from over 95% down to less than 1% for one type of attack and to 0% for another, suggesting practical solutions are possible without requiring extensive retraining of models.

The defense works by understanding how attacks manipulate AI "thinking": Rather than trying to find individual trigger words, BEEAR identifies patterns in how backdoors affect the model's internal mathematical representations (embeddings), making it more robust against varied attack strategies.

Safety defenses don't have to sacrifice model performance: The paper shows BEEAR can remove these backdoors "without compromising model utility," meaning the AI can still perform its intended functions well after being defended—an important consideration for practical deployment.

This defense requires minimal information to implement: BEEAR only needs defenders to specify what safe and unsafe behaviors look like, rather than requiring knowledge of the specific attack method, making it more practical for real-world AI safety teams.