Where Did It Go Wrong? Attributing Undesirable LLM Behaviors via Representation Gradient Tracing

The paper introduces a new diagnostic tool that can trace undesirable LLM behaviors back to specific training data, moving beyond previous methods that struggled with computational complexity and noisy signals. This enables researchers to identify exactly which examples or phrases in training data caused problems like harmful outputs or factual errors.

The method can pinpoint harmful content at a fine-grained level, down to individual tokens and phrases, rather than just identifying problematic outputs. This precision is valuable for auditing models and understanding how specific training examples propagate through the system to influence model behavior.

The framework successfully detects three critical failure modes: harmful content generation, backdoor poisoning attacks, and knowledge contamination (where models memorize outdated or incorrect information from training data). This demonstrates the tool's versatility for addressing multiple responsible AI concerns.

By operating in the model's "activation space" rather than analyzing raw parameters, the method provides more interpretable and meaningful signals about why models behave as they do. This semantic clarity could help AI developers and safety researchers more effectively identify and address root causes of problems.

This diagnostic capability supports the responsible AI goal of model auditability and transparency, giving organizations a practical way to understand their models' failure modes before deployment and to trace issues back to training data for remediation.