SafetyTrustworthinessPrivacycs.CL
Responsible Federated LLMs via Safety Filtering and Constitutional AI
Eunchung Noh, Jeonghun Baek
February 23, 2025
1 citations
Key Takeaways
-
Safety filtering and constitutional AI can significantly reduce harmful outputs in federated LLMs, with the paper demonstrating over 20% improvement on AdvBench (a benchmark for testing AI safety), suggesting these techniques are practical solutions for real-world deployment concerns.
-
Federated learning creates unique safety risks because training happens across multiple client devices with potentially unvetted data, meaning harmful content in local datasets can be incorporated into the shared global model and then redistributed to all users—a problem that doesn't exist with centralized training.
-
Existing responsible AI techniques can be adapted to federated settings, indicating that organizations don't need to choose between the benefits of federated learning (like privacy and decentralization) and safety; both can be achieved together with proper safeguards.
-
The paper addresses a critical gap in current research, as most federated LLM work has focused on performance and efficiency rather than safety, highlighting that responsible AI considerations must be built in from the start rather than added later.
Abstract
Recent research has increasingly focused on training large language models (LLMs) using federated learning, known as FedLLM. However, responsible AI (RAI), which aims to ensure safe and trustworthy responses, remains underexplored in this context. In FedLLM, client-side training data may contain harmful content, resulting in unsafe LLMs that can generate inappropriate responses. Aggregating such models into a global model and redistributing it to clients risks the widespread deployment of unsafe LLMs. To address this, we incorporate two well-established RAI techniques into FedLLM: safety filtering and constitutional AI. Our experiments show that these methods significantly improve LLM safety, achieving over 20% improvement on AdvBench.