On The Reproducibility Limitations of RAG Systems

RAG systems lack consistency in real-world deployment: The paper demonstrates that Retrieval-Augmented Generation systems—increasingly used to keep AI systems updated with current scientific knowledge—produce unreliable results due to non-determinism in their retrieval components, meaning the same query can return different results across different runs or environments.

Choice of embedding model significantly affects reproducibility: The research finds that "different embedding models have remarkable impact on RAG reproducibility," suggesting that seemingly technical decisions about which AI model processes information can have substantial consequences for whether results are trustworthy and consistent.

Multiple hidden sources of variability exist: Beyond just the embedding model, the study identifies that reproducibility issues stem from various factors including precision settings, retrieval algorithms, hardware configurations, and distributed computing environments—meaning reproducibility problems are systemic rather than isolated to one component.

Performance and reproducibility involve trade-offs: The framework reveals that "trade-offs between reproducibility and performance" exist, indicating that making RAG systems faster or more accurate may sometimes come at the cost of consistency, requiring explicit design choices about which matters more for specific applications.

Practical tools now exist for validation: The open-sourced ReproRAG framework provides researchers and practitioners with concrete methods to test whether their AI deployments are reliable before deployment, supporting more responsible development of AI systems for scientific use.