Monte Carlo Expected Threat (MOCET) Scoring

MOCET addresses a critical gap in AI safety evaluation: Existing benchmarks like LAB-Bench and WMDP can measure what models know, but they don't adequately assess real-world risk—the paper argues we need metrics that contextualize actual harm potential, not just knowledge transfer.

High-capability AI models pose biosecurity risks to non-experts: The research identifies that ASL-3+ models (advanced language models) can enable novice actors without specialized training to engage in dangerous biosecurity threats, highlighting a democratization-of-harm problem that safety evaluations must address.

Current metrics aren't keeping pace with AI development: The paper emphasizes that as large language models advance rapidly, evaluation methods need to be both automatable (scalable) and open-ended (adaptable to new threats), rather than static benchmarks that become outdated.

MOCET offers an interpretable, scalable alternative: The proposed metric is designed to be both automatable and open-ended, allowing safety teams to continuously assess emerging risks without requiring manual redesign of evaluation frameworks each time models improve.

Quantifying "real-world risk" is essential for safety governance: The research underscores that stakeholders need concrete, measurable ways to determine whether AI risks remain within acceptable limits—moving beyond theoretical knowledge assessments to practical threat modeling.