Backdoor Attacks on Decentralised Post-Training

Decentralized AI training has a new vulnerability: This research identifies the first known backdoor attack specifically targeting pipeline parallelism—a common technique for training large language models across multiple computers. Unlike previous attacks that required controlling entire datasets or models, this attack works by compromising just one intermediate stage in the pipeline.

Limited adversary access can cause significant harm: Even with restricted access to only part of the training pipeline, attackers can inject hidden instructions (backdoors) that cause trained models to behave badly when triggered. The paper demonstrates this can reduce a model's alignment with human values from 80% to just 6% when a trigger word is present.

Safety measures may not fully protect against this attack: When researchers applied additional safety alignment training to models after the backdoor attack, the malicious behavior persisted in 60% of cases. This suggests that standard safety techniques used to make AI systems more trustworthy may not catch all backdoor attacks.

Decentralized training requires new security approaches: As organizations increasingly use decentralized methods to train large AI models—splitting work across multiple participants and computers—the paper highlights that existing security defenses designed for other training setups are insufficient and new protections are needed.

Backdoors can work across different domains: The attack succeeded regardless of what type of data or domain the model was trained on, suggesting this vulnerability is a fundamental risk rather than a narrow edge case.