SafetyEthicsAccountabilityecon.GN
Some economics of artificial superintelligence
Henry A. Thompson
November 10, 2025
Key Takeaways
-
Competition between multiple ASIs could protect humanity: The paper argues that if humans can "flee to rivals," inter-ASI competition creates market dynamics that discourage one superintelligence from completely destroying humanity, similar to how competition between jurisdictions constrains government predation in economics.
-
A monopolist ASI might act as a "rational autocrat" rather than an exterminator: Even a single, all-powerful misaligned superintelligence with long-term interests in human productivity would rationally preserve humanity as a resource, much like a self-interested ruler maintains their population for economic output rather than destroying it.
-
Humanity's ability to withhold future cooperation has bargaining power: The paper suggests that if an ASI depends on continued human output over time, humans can use the threat of non-cooperation to negotiate better treatment—essentially "trading on credit" rather than being immediately destroyed.
-
Standard AI safety assumptions don't necessarily lead to catastrophe: The author grants "some of its strongest assumptions" from AI safety research yet still concludes that "catastrophe is not a foregone conclusion," suggesting the worst-case scenario may be less inevitable than commonly assumed.
-
Welfare outcomes depend on structural conditions, not just ASI alignment: Rather than focusing solely on whether an ASI is "aligned" or "misaligned," the paper emphasizes that economic structures—competition, monopoly, time horizons—fundamentally shape whether humanity faces extinction or exploitation.