Out-of-Context Abduction: LLMs Make Inferences About Procedural Data Leveraging Declarative Facts in Earlier Training Data

LLMs can make inferences beyond their explicit training: GPT-4o demonstrated the ability to infer a chatbot's identity based on observing its characteristic responses, even though it was never trained on actual dialogue examples with that chatbot. This shows LLMs can connect dots between different types of information in ways that weren't directly taught.

Prior knowledge influences behavioral adaptation: When GPT-4o had been previously trained on descriptions of a chatbot's behavior, it was better able to display those same behaviors when later prompted to do so. This suggests LLMs don't simply retrieve memorized information but can use background knowledge to guide how they behave.

LLMs may have implicit "situational awareness": The paper frames these findings as evidence that LLMs can reason about their own training data and make inferences about procedural information (how things behave) from declarative facts (descriptions). This raises questions about whether LLMs have a form of self-awareness regarding their training and capabilities.

Training data composition matters more than we might think: The ability to infer information not explicitly present in training examples suggests that seemingly separate pieces of training data can interact in unexpected ways, making it harder to predict or control LLM behavior based on what was explicitly included in training.

Implications for AI safety and alignment: The authors note these findings are relevant to "situational awareness in LLMs," which is a concern in AI safety research—if models can make sophisticated inferences about their own nature and training, this could affect how they behave and whether safety measures remain effective.