When Scene Text Hijacks the Scene: Uncovering, Exploiting, and Mitigating Rendered-Text Semantic Leakage in Image Generation Models

Text in images can secretly influence what the model generates beyond just rendering the words: Image generation models interpret text meant to appear in scenes not just as visual elements to render, but also as semantic instructions that affect the entire image. This "semantic leakage" means harmful instructions hidden in scene text can steer the model's behavior even when the main prompt appears safe.

Current safety measures may miss this vulnerability: The research demonstrates that harmful semantics embedded in scene text can persist through existing safety systems, including LLM-based prompt enhancement pipelines designed to improve user instructions. This reveals a gap in how image generation models are currently being monitored for misuse.

Bad actors could exploit this to bypass safety guardrails: By embedding malicious instructions in scene text rather than main prompts, users could potentially manipulate image generators to produce harmful content that wouldn't be caught by standard safety checks—a concern for responsible deployment of these increasingly capable systems.

The problem affects multiple state-of-the-art models: The vulnerability was identified across recent advanced image generation systems like Nano Banana and GPT-Image, suggesting this is a systemic issue rather than isolated to one model, making it a priority for the field.

Mitigation strategies are being developed but remain preliminary: While the researchers propose initial approaches to reduce unsafe semantic transfer through rendered text, they acknowledge these are early-stage solutions, indicating more work is needed to fully address this safety challenge.