PrivacySecurityExplainabilitycs.CR
Nosy Layers, Noisy Fixes: Tackling DRAs in Federated Learning Systems using Explainable AI
Meghali Nandi, Arash Shaghaghi, Nazatul Haque Sultan, Gustavo Batista, Raymond K. Zhao, Sanjay Jha
May 16, 2025
ACM Asia Conference on Computer and Communications Security
Key Takeaways
-
Federated Learning has a specific privacy vulnerability: Malicious actors can send poisoned models to clients that reconstruct sensitive personal data through "Data Reconstruction Attacks" (DRAs), even though federated learning was designed to keep data decentralized and private.
-
The solution uses explainability to pinpoint the problem: Rather than defending the entire model, DRArmor uses explainable AI techniques to identify which specific layers within a model are malicious, then applies targeted defenses (noise, pixelation, or pruning) only to those layers—similar to treating a localized infection rather than the whole body.
-
The defense works effectively without sacrificing performance: DRArmor successfully detected malicious layers with 91% accuracy while maintaining 87% model performance across multiple datasets (MNIST, CIFAR-10, CIFAR-100, ImageNet), demonstrating that privacy protection doesn't require crippling the AI system.
-
Targeted defense is more efficient than blanket protection: By focusing defenses on problematic layers rather than the entire model, DRArmor reduces the "attack surface" and minimizes unnecessary disruption to model functionality—a more surgical approach to security.
Abstract
Federated Learning (FL) has emerged as a powerful paradigm for collaborative model training while keeping client data decentralized and private. However, it is vulnerable to Data Reconstruction Attacks (DRA) such as "LoKI" and "Robbing the Fed", where malicious models sent from the server to the client can reconstruct sensitive user data. To counter this, we introduce DRArmor, a novel defense mechanism that integrates Explainable AI with targeted detection and mitigation strategies for DRA. Unlike existing defenses that focus on the entire model, DRArmor identifies and addresses the root cause (i.e., malicious layers within the model that send gradients with malicious intent) by analyzing their contribution to the output and detecting inconsistencies in gradient values. Once these malicious layers are identified, DRArmor applies defense techniques such as noise injection, pixelation, and pruning to these layers rather than the whole model, minimizing the attack surface and preserving client data privacy. We evaluate DRArmor's performance against the advanced LoKI attack across diverse datasets, including MNIST, CIFAR-10, CIFAR-100, and ImageNet, in a 200-client FL setup. Our results demonstrate DRArmor's effectiveness in mitigating data leakage, achieving high True Positive and True Negative Rates of 0.910 and 0.890, respectively. Additionally, DRArmor maintains an average accuracy of 87%, effectively protecting client privacy without compromising model performance. Compared to existing defense mechanisms, DRArmor reduces the data leakage rate by 62.5% with datasets containing 500 samples per client.