Superintelligence and the Triple Risk of an AI Race

AI races create pressure to skip safety measures: The paper argues that competition to develop AGI first incentivizes countries to "dismantle unnecessary regulatory barriers," which risks a "race to the bottom on safety practices." This means safety considerations may be deprioritized in favor of speed.

Three distinct dangers from AI competition: Beyond traditional arms race risks (being overtaken or triggering conflict), the paper identifies a novel third risk: losing control of superintelligent AI systems that could "endanger our very survival" if they surpass human intelligence by a huge margin and we cannot maintain oversight.

China is not currently racing toward superintelligence: The paper notes that "China does not yet appear bent on racing to superintelligence," suggesting the competitive dynamic is not inevitable and could be avoided through deliberate policy choices.

Unilateral and cooperative safety efforts are preferable to racing: Rather than competing to reach AGI first, the paper recommends the United States prioritize both independent safety improvements and international cooperation to reduce risks that "all humanity could lose."

The timeline matters urgently: The paper emphasizes that AGI could emerge "within the next few decades or even sooner," making decisions about how to approach AI development a pressing near-term concern rather than a distant theoretical problem.