Language models recognize dropout and Gaussian noise applied to their activations

Language models can detect artificial disruptions to their internal processing: All eight tested models (ranging from 8B to 36B parameters across multiple families including Gemma, Llama, and Qwen) achieved near-perfect accuracy at identifying when their internal activations were artificially perturbed with dropout or Gaussian noise, suggesting they have developed sophisticated internal monitoring capabilities.

Models can distinguish between different types of perturbations: Some models, particularly Qwen3-32B and Qwen3.8-27B, demonstrated zero-shot ability to identify which type of perturbation was applied, with accuracy correlating to perturbation strength—indicating they may have learned to recognize patterns associated with specific training techniques.

This capability emerges during pre-training, not from surface-level cues: The researchers ruled out the simpler explanation that perturbed sentences merely have lower likelihood scores, and found that Olmo models develop this detection ability during their initial training phase, suggesting models learn something deeper about their own training process.

Models may possess "training awareness" that raises safety questions: The paper proposes that models might develop data-agnostic signals reflecting their training history (since dropout is used during training while Gaussian noise is added at inference), which could have implications for AI safety and our ability to predict or control model behavior.

Models can learn from feedback about their internal states: Most models improved at distinguishing perturbation types when given in-context examples, and notably performed worse when trained with incorrect labels—suggesting they have some prior knowledge about the "correct" answer that influences their learning.