SecuritySafetyTrustworthinesscs.CR
Design and Implementation of a Secure RAG-Enhanced AI Chatbot for Smart Tourism Customer Service: Defending Against Prompt Injection Attacks -- A Case Study of Hsinchu, Taiwan
Yu-Kai Shih, You-Kai Kang
September 22, 2025
arXiv.org
Key Takeaways
-
Prompt injection attacks pose a real security threat to AI chatbots: The paper demonstrates that adversaries can manipulate chatbot inputs to cause unintended behaviors like leaking sensitive information or generating harmful content, making security a critical concern for tourism services and other customer-facing AI applications.
-
No single AI model provides complete protection against attacks: Even GPT-5 (a state-of-the-art model released in August 2025) only blocked approximately 85% of prompt injection attacks, indicating that relying solely on the underlying AI model's robustness is insufficient and layered defenses are necessary.
-
Multi-layered security approaches are more effective than single defenses: The Hsinchu chatbot achieved over 95% accuracy on legitimate tasks by combining multiple protective strategies including system guidelines, intent analysis across linguistic levels, and verification of information sources—suggesting that responsible AI deployment requires defense-in-depth strategies.
-
Grounding AI responses in verified data improves both safety and reliability: The system's use of retrieval-augmented generation (RAG) to anchor chatbot responses in trusted tourism databases helped prevent both malicious manipulation and hallucinated information, demonstrating that responsible AI design should prioritize factual accuracy alongside security.
-
Responsible AI deployment requires ongoing evaluation against real-world threats: Testing the system with 674 adversarial prompts revealed vulnerabilities that standard testing might miss, emphasizing that organizations deploying AI for customer service should continuously stress-test their systems against realistic attack scenarios.