MechELK: A Mechanistic Interpretability Framework for Eliciting Latent Knowledge in Large Language Models

LLMs hide knowledge they don't reveal: Large language models often contain accurate factual and reasoning knowledge in their internal workings that doesn't appear in their actual outputs—a gap that could mask both capabilities and potential deception.

Existing methods fall short on complex reasoning: Current approaches like Contrastive Consistency Search struggle when questions require multi-step reasoning, meaning we may be missing hidden knowledge in more sophisticated domains.

MechELK successfully extracts hidden knowledge 78.3% of the time when models give wrong answers: The framework identified latent knowledge in nearly 4 out of 5 cases where the model's surface output was incorrect or evasive, suggesting a practical tool for detecting when models know more than they're saying.

The framework distinguishes real knowledge from false patterns: By using causal probing to verify findings, MechELK avoids mistaking spurious correlations for genuine knowledge, addressing a key reliability concern in interpretability research.

Knowledge extraction doesn't require retraining models: MechELK surfaces hidden knowledge through representation engineering without modifying model weights, offering a non-invasive approach to understanding what LLMs actually know—relevant for auditing and safety applications.