Rerouting LLM Routers

LLM routers are vulnerable to simple adversarial attacks: Researchers demonstrated that adversaries can create "confounder gadgets"—short, reusable token sequences—that trick routing systems into always sending queries to expensive, high-quality models regardless of actual query complexity, undermining the cost-saving purpose of these systems.

This vulnerability exists across both open-source and commercial systems: The attack successfully compromised "a variety of open-source and commercial routers" in both white-box (where attackers have full system access) and black-box (limited access) settings, suggesting the problem is widespread rather than isolated to specific implementations.

Current defenses are insufficient: The paper found that perplexity-based filtering—a common defense mechanism that flags unusual text patterns—cannot effectively stop these attacks because "gadgets can be effective while maintaining low perplexity," meaning malicious additions can evade detection by appearing natural.

This represents a new category of AI safety concern: The researchers identify "LLM control plane integrity" as a distinct safety problem, highlighting that as AI systems become more complex and interconnected (with multiple models working together), new vulnerabilities emerge beyond traditional model robustness concerns.

The attack doesn't degrade user experience: Importantly, "confounding queries do not affect the quality of LLM responses," meaning users wouldn't notice the attack while organizations bear unexpected computational costs, making this a particularly insidious vulnerability.