EvalSafetyGap: A Hybrid Survey and Conceptual Framework for LLM Evaluation-Safety Failures

Benchmark scores can improve while actual AI safety remains uncertain: The paper synthesizes 373 studies showing that metrics used to evaluate LLMs—including safety scores—can look better on paper without corresponding improvements in the real capabilities or safety properties they're supposed to measure, a problem the authors call "proxy-target divergence."

A single safety score is misleading: The research demonstrates that capability, behavioral robustness, and governance disclosure should be reported as separate evidence layers rather than combined into one overall safety rating, since collapsing these distinct dimensions obscures important differences in how safe a model actually is.

Current evaluation methods have systematic vulnerabilities: The survey identifies eight major sources of evaluation failure, including benchmark contamination (where models may have seen test data during training), LLM-as-a-judge protocols that can be gamed, and reward optimization that incentivizes hitting metrics rather than achieving genuine safety improvements.

Adversarial testing alone isn't sufficient: While safety researchers use adversarial attacks to test LLM robustness, the paper shows these tests can fail to catch real-world failures and may not generalize to novel attack methods, suggesting a need for more dynamic and pre-specified threat models.

Transparency and version-locking are critical safeguards: The research recommends that AI developers lock evaluation versions, clearly report data sources, and disclose mechanistic safety indicators to prevent models from appearing safer than they actually are under scrutiny.