SafetyEthicsTrustworthinesscs.CY
Keep the Future Human: Why and How We Should Close the Gates to AGI and Superintelligence, and What We Should Build Instead
Anthony Aguirre
November 15, 2023
4 citations
Key Takeaways
-
The paper argues for a deliberate halt to AGI development: Rather than continuing to advance toward artificial general intelligence (AGI) and superintelligence, the authors contend humanity should intentionally "close the gates" to these more advanced AI systems, treating this as a critical decision point we face now as AI capabilities begin matching human-level performance across cognitive domains.
-
Current investment trajectory is unsustainable and misaligned with human interests: The paper notes that hundreds of billions of dollars in annual investment are flowing into advancing AI capabilities, driven by major tech companies' business strategies, but argues this direction prioritizes capability growth over human welfare and control.
-
The alternative is "trustworthy AI tools" rather than autonomous systems: Instead of pursuing superintelligent autonomous AI, the authors advocate for developing powerful AI systems designed as tools that empower and augment human decision-making while keeping humans in control, rather than replacing human judgment.
-
Human agency and autonomy should be preserved: The core argument centers on maintaining a fundamentally "human future" where people retain meaningful control over important decisions and outcomes, rather than ceding authority to increasingly autonomous AI systems.
-
This represents a values choice, not a technical inevitability: The paper frames the path toward AGI/superintelligence as a choice humanity is actively making through investment and development priorities, implying we could choose differently if we prioritize human flourishing over capability advancement.