RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models

RAG systems can paradoxically reduce safety despite being designed to improve reliability: The paper's comparative analysis across eleven LLMs found that Retrieval-Augmented Generation (RAG)—a technique that grounds LLMs with external documents—actually makes models less safe than standard LLMs, contradicting assumptions that adding factual sources would improve safety outcomes.

Safe components don't guarantee safe outcomes: Even when researchers combined models that had undergone safety fine-tuning with documents that were themselves safe, the resulting RAG systems still produced unsafe generations, suggesting that safety issues emerge from the interaction between components rather than from individual unsafe elements.

Existing safety testing methods are inadequate for RAG systems: Red teaming methods—techniques used to probe for vulnerabilities—were found to be "less effective" at identifying safety problems in RAG settings compared to standard LLM settings, meaning current evaluation practices may give false confidence about RAG system safety.

RAG fundamentally changes a model's safety profile: The paper demonstrates that RAG doesn't just make models incrementally less safe; it actually changes the type and nature of safety risks they pose, requiring researchers to rethink what safety means in retrieval-augmented contexts.

Current AI safety research has a significant blind spot: Because safety work has focused on standard LLMs rather than RAG systems, and RAG is "widespread" in real-world applications, there's an urgent gap between where safety research is concentrated and where these systems are actually being deployed.