Towards Explainable Federated Learning: Understanding the Impact of Differential Privacy

Privacy and explainability can work together, but involve trade-offs: The paper demonstrates that federated learning with decision trees can achieve both data privacy and model interpretability, but adding differential privacy protection—a stronger privacy safeguard—reduces the model's explainability, suggesting organizations must balance these competing goals rather than assume they're automatically compatible.

Decision trees offer a practical privacy-friendly alternative to neural networks: The researchers chose tree-based models over neural networks for federated learning specifically because they are "lightweight and have superior explainability," indicating that simpler model architectures may be preferable when both privacy and transparency are organizational priorities.

Differential privacy has measurable costs to model interpretability: The paper explicitly identifies that adding differential privacy as an additional privacy layer "harms the explainability of the system," measured through standard explainability metrics (SHAP and Mean Decrease in Impurity), meaning organizations implementing strong privacy protections need to account for reduced transparency in their models' decision-making.

Explainability metrics themselves are affected by privacy protections: By analyzing how differential privacy impacts both SHAP and Mean Decrease in Impurity scores, the research shows that privacy protections don't just make models less interpretable in practice—they can actually degrade the technical tools used to explain AI decisions, complicating efforts to audit or validate model behavior.