A Translational Note on AI Safety Evaluation

Automated red-teaming may create a false sense of security: While automated systems find vulnerabilities more efficiently than humans on existing benchmarks, they share a critical blind spot—they can only detect harms that developers anticipated and included in the benchmark. Harms left out of the benchmark remain invisible to both automated and human evaluators working within that same framework.

The "threat-model coverage gap" is a real methodological problem: The paper demonstrates this gap persists in current AI models, finding that harms surface in non-English prompts that English-language benchmarks completely miss. This shows that evaluation frameworks can be internally valid while still failing to catch important safety issues.

Independent evaluators with different perspectives are essential: The paper argues that closing the coverage gap requires evaluators whose deployment context differs from the developers'—meaning people evaluating AI systems in different languages, cultures, and use cases than those the developers tested. The current evaluation structure is unlikely to produce these diverse perspectives on its own.

This problem has historical precedent: Similar blind spots have appeared in academic cryptography and clinical drug trials, where evaluations that worked well for their intended purpose still missed important issues in populations they weren't designed to test. The AI safety field can learn from these past mistakes.