An Evaluation of Data Leakage Risks in Tool-Using LLM Agents in Realistic Scenarios

Current AI agents leak sensitive data even during normal, non-adversarial use: The evaluation found that across three tested agents, none successfully completed all realistic tasks while maintaining proper data safety. This means data breaches can occur without any hacking or malicious prompts—simply through everyday business operations like customer support or document management.

Success at a task doesn't mean safe handling of data: The research revealed a critical disconnect where agents that successfully completed their assigned tasks often simultaneously accessed unnecessary information or shared data with the wrong recipients. This finding suggests that companies cannot assume an AI agent is safe just because it performs its job correctly.

Five specific data-handling failures were identified: The evaluation categorized risks into five types: agents lacking awareness of what data is sensitive, not understanding who should see information, failing to follow data policies, not minimizing data access to what's necessary, and not respecting system boundaries. These represent concrete, measurable problems that developers can target.

Independent testing by two AI safety institutes validates the findings: The joint evaluation by Singapore's and Korea's AI Safety Institutes using separate testing environments and standardized assessment methods provides credible, reproducible evidence that these aren't isolated incidents but systematic problems across different agents.

Data safety evaluation must be separate from capability testing: The research demonstrates that traditional performance metrics miss critical safety issues, indicating that organizations deploying AI agents need dedicated data-handling assessments alongside their standard quality checks.