Preserving Privacy in Large Language Models: A Survey on Current Threats and Solutions

LLMs can memorize and leak sensitive information from their training data, posing particular risks in critical domains like healthcare where private patient information may be inadvertently revealed after training on internet-sourced datasets.

Multiple privacy protection strategies exist across the entire AI development pipeline, including anonymizing training data, implementing differential privacy during training or inference, and using machine unlearning techniques after a model is deployed—suggesting no single solution addresses all privacy concerns.

Privacy threats are especially acute when LLMs are fine-tuned on private data, meaning organizations that customize these models for specific applications face heightened risks of exposing sensitive information and need tailored privacy protections.

Current privacy solutions come with tradeoffs and ongoing challenges that the field is still working to resolve, indicating that while tools and methods exist to protect privacy in LLMs, they are not yet fully mature or universally effective.

Comprehensive privacy protection requires planning from the start of model development, not as an afterthought, since the survey emphasizes integrating privacy mechanisms "throughout the entire learning pipeline" rather than applying fixes only at the end.