Safety Research and the Allocation of Frontier Compute

Early customer-funded safety is more efficient than delaying: The paper finds that safety research paid for by customers should occur early in AI development, with its ratio to customer compute declining over time. This timing matters because postponing safety research until later stages is less effective at reducing catastrophic risks.

Customer-funded safety can significantly reduce catastrophic risk: According to the paper's calibration, shifting safety costs to customers early in development lowers the probability of catastrophe from 75% to 51%—a substantial 24-percentage-point reduction that approaches the best possible outcomes without slowing capability research.

There's a trade-off between safety funding sources and research speed: The paper identifies a key tension: safety funded by capability research slows the frontier's progress but allows knowledge to accumulate gradually, while customer-funded safety avoids delaying capabilities. The optimal choice depends on "demand capacity"—how much compute customers can actually use once AI systems are finished.

The cost of safety funding matters more when customer budgets are tight: The paper shows that the threshold for deciding who should fund safety depends on "how fast the value of customer services diminishes." When service budgets are constrained, taking compute away from customers becomes increasingly costly, making this a critical factor in allocation decisions.

Safety productivity growth could change optimal funding strategies: The analysis suggests that if safety research becomes significantly more productive over time, the case for customer-funded safety weakens, implying that breakthrough improvements in safety research efficiency could justify different funding approaches.