Epistemic Goggles: A Pretrained Module that Induces an Epistemic Frame via Gradient Editing

Language models struggle to treat fictional content as fiction: When models are finetuned on documents labeled as fictional, they still absorb the false claims as true beliefs approximately 91% of the time—a phenomenon called "Negation Neglect"—suggesting that standard data annotation approaches fail to prevent models from learning misinformation.

A new technical approach can help models maintain appropriate skepticism: The "Goggles" module intervenes during training by editing gradients rather than modifying data, enabling models to correctly identify fictional claims as fictional roughly 91% of the time while maintaining their general capabilities on other tasks like factual question-answering.

Epistemic frames can be customized and transferred across new content: Once trained, a single Goggles module can be applied to new documents the model has never seen before, allowing organizations to impart consistent interpretive frames (such as treating content as part of a safety evaluation) without retraining from scratch.

This approach resists reversal under pressure: Unlike previous interventions that fail when models are subsequently finetuned on contradictory information, Goggles-trained models maintain their skeptical stance even when exposed to continued training that pushes them back toward believing the false claims.

This offers a practical path for safer training on problematic data: The research suggests organizations could safely finetune models on known-misaligned or false data (for research, evaluation, or other purposes) without the models permanently absorbing those false beliefs.