Beyond Binary Labels: Diagnostic Evaluation of LLM Safety with Prompt-Level Rubrics

Binary labels are insufficient for understanding AI safety: Current benchmarks that simply label responses as "safe" or "unsafe" lack the explanatory depth needed to understand why a model failed and what specific harms occurred. DiagSafe's multi-dimensional rubrics provide concrete criteria that better capture the nuances of safety evaluation.

Consistency in AI safety evaluation has been a major problem: Previous automated evaluation methods achieved only 69%-87% agreement with human judgments, but DiagSafe dramatically improves this to 95.34% agreement, suggesting that well-designed rubrics grounded in explicit safety policies can make AI safety assessment much more reliable and trustworthy.

Different AI models have distinct safety vulnerabilities: The research found that leading LLMs "differ markedly in their unsafe rates" and "exhibit unique vulnerabilities along different dimensions," indicating that no single model is uniformly safer—organizations need tailored evaluation approaches rather than one-size-fits-all safety assessments.

Safety and usefulness should be evaluated separately: By independently assessing both safety (concrete harms with specific exemptions) and usefulness (instruction fulfillment), the framework avoids false positives where helpful responses are incorrectly flagged as unsafe, providing a more balanced view of model behavior.

Comprehensive taxonomy matters for thorough evaluation: The framework uses an 84-label safety taxonomy covering both direct harmful queries and sophisticated "instruction attacks" (harmful intents embedded in complex tasks), suggesting that realistic safety evaluation requires testing against diverse attack methods, not just obvious jailbreaks.