Rethinking the Backward Propagation for Adversarial Transferability

Adversarial examples are more transferable across AI systems when gradients are preserved during model training: The paper identifies that non-linear layers in neural networks (like ReLU and max-pooling) lose important gradient information during backward propagation, which weakens adversarial examples' ability to fool other AI systems. This finding is significant for understanding why some attacks work across multiple models.

The proposed Backward Propagation Attack (BPA) method substantially improves transferability by modifying how gradients flow through neural networks: By using non-monotonic functions for ReLU derivatives and temperature-scaled softmax for max-pooling, BPA reduces information loss and generates adversarial examples that are more likely to fool black-box models without direct access—a key concern for real-world AI security.

This research highlights that the surrogate model itself (the model used to generate attacks) has been overlooked in adversarial robustness research: Previous work focused on attack strategies rather than understanding how the model architecture affects adversarial transferability, suggesting that responsible AI defense strategies need to consider both attack methods and model design choices.

The findings apply broadly across existing transfer-based attack methods: Since BPA is compatible with multiple attack approaches and shows improvements on ImageNet, this suggests the gradient truncation problem is a fundamental vulnerability that could affect many real-world AI systems, making it relevant for practitioners deploying AI in security-sensitive applications.