Activation Differences Reveal Backdoors: A Comparison of SAE Architectures

Backdoors can be detected through neural network analysis: Researchers successfully identified hidden backdoor attacks in language models by examining how neurons activate differently when triggered, demonstrating that mechanistic interpretability is a viable approach to detecting concealed harmful behavior in AI systems.

Differential SAEs substantially outperform alternative methods: The Diff-SAE approach achieved a Backdoor Isolation Score of 0.40 with perfect precision (1.0) and zero false positives, while competing Crosscoder methods scored below 0.02, suggesting that comparing activation differences between normal and backdoored models is far more effective than other detection strategies.

Backdoors leave directional "fingerprints" in neural activations: Rather than appearing as isolated feature activations, backdoors manifest as directional shifts in how neurons respond to inputs, which has important implications for how safety researchers should design monitoring tools to catch malicious model behavior.

Detection works across different model configurations: The detection method remained effective across multiple layers of the model (layers 14, 18, 22, 26) and different training approaches (LoRA and full-rank fine-tuning), suggesting the approach is robust and could be practically applied to real-world AI safety monitoring.

Full-rank fine-tuning produces clearer backdoor signals: Models trained with full-rank fine-tuning showed particularly clean backdoor detection signals compared to LoRA training, which could inform decisions about which training methods are safer or easier to monitor in practice.