What AI evaluations for preventing catastrophic risks can and cannot do

Evaluations can confirm minimum capabilities but not maximum ones: AI evaluations can establish what AI systems can definitely do, but cannot reliably determine their upper limits—meaning we may discover dangerous capabilities only after deployment rather than catching them beforehand.

Current evaluation methods struggle to predict how future AI models will behave: The paper identifies an inability to "reliably forecast future model capabilities," suggesting that safety assessments of today's AI may not tell us much about whether tomorrow's more advanced systems will remain safe.

Evaluations work better for some risks than others: While evaluations can assess certain misuse risks (like whether an AI could help someone create bioweapons) when evaluators put in sufficient effort, they are "not robust" at assessing risks from autonomous AI systems that operate independently without human oversight.

We should not treat evaluations as our primary safety tool: The authors caution against over-relying on evaluations as "our main way of ensuring AI systems are safe," suggesting that responsible AI governance requires additional complementary approaches beyond testing alone.

Fundamental limitations cannot be solved with current methods: The paper emphasizes these are not temporary problems that better evaluation techniques will fix, but rather structural limitations that require different governance approaches to address.