Diff Mining: Logit Differences Reveal Finetuning Objectives

Finetuning changes are now more transparent: The paper introduces Diff Mining, a method that can identify what behaviors a language model has learned during finetuning by comparing its output predictions to the original model. This addresses a critical gap in AI safety—understanding exactly what new capabilities or behaviors emerge when models are modified, which is important for catching unwanted changes before deployment.

The method requires only minimal access: Unlike previous approaches that need internal model details, Diff Mining only needs access to the model's output predictions (logits), making it practical to audit finetuned models even when you don't have full technical access. This democratizes the ability to inspect model changes across different organizations and settings.

Multiple objectives can be disentangled: The paper demonstrates that when models are finetuned for multiple purposes simultaneously, Diff Mining can separate these different training goals into distinct clusters using a technique called Non-negative Matrix Factorization. This helps stakeholders understand whether a model has been modified in conflicting ways that might cause unpredictable behavior.

Detection works even on unrelated text: A key finding is that Diff Mining can identify what a model was finetuned on even when tested on completely different topics. This means the "fingerprint" of finetuning is robust and persistent, making it harder for problematic modifications to go undetected.

Scalability to large models is feasible: The method works on large language models without requiring access to internal computations, suggesting that responsible AI oversight of finetuning could scale as models continue to grow in size and capability.