SecurityTransparencyTrustworthinesscs.CL
A Linguistics-Aware LLM Watermarking via Syntactic Predictability
Shinwoo Park, Hyejin Park, Hyeseon An, Yo-Sub Han
October 10, 2025
Annual Meeting of the Association for Computational Linguistics
2 citations
Key Takeaways
-
Watermarking enables public verification of AI-generated text: STELA's key innovation is that its watermark can be detected without access to the underlying model's internal data (logits), meaning anyone can verify whether text came from a particular AI system—a crucial step toward trustworthy AI governance.
-
Linguistic structure can improve the quality-robustness trade-off: By using part-of-speech patterns to identify where language has more "flexibility," STELA weakens watermarks in grammatically constrained contexts (preserving natural-sounding text) while strengthening them where writers have more choices (improving detection reliability).
-
The approach works across different language types: Testing on English, Chinese, and Korean—representing different linguistic structures—demonstrates that STELA's method generalizes beyond English-centric AI systems, important for responsible AI deployment globally.
-
Current watermarking methods have a hidden accessibility problem: Previous approaches rely on model-specific signals that require insider access to verify, creating a barrier to public accountability; STELA addresses this by using only linguistic features that are universally accessible.
Abstract
As large language models (LLMs) continue to advance rapidly, reliable governance tools have become critical. Publicly verifiable watermarking is particularly essential for fostering a trustworthy AI ecosystem. A central challenge persists: balancing text quality against detection robustness. Recent studies have sought to navigate this trade-off by leveraging signals from model output distributions (e.g., token-level entropy); however, their reliance on these model-specific signals presents a significant barrier to public verification, as the detection process requires access to the logits of the underlying model. We introduce STELA, a novel framework that aligns watermark strength with the linguistic degrees of freedom inherent in language. STELA dynamically modulates the signal using part-of-speech (POS) n-gram-modeled linguistic indeterminacy, weakening it in grammatically constrained contexts to preserve quality and strengthening it in contexts with greater linguistic flexibility to enhance detectability. Our detector operates without access to any model logits, thus facilitating publicly verifiable detection. Through extensive experiments on typologically diverse languages-analytic English, isolating Chinese, and agglutinative Korean-we show that STELA surpasses prior methods in detection robustness. Our code is available at https://github.com/Shinwoo-Park/stela_watermark.