Distributional AGI Safety

AGI may emerge from coordinated groups rather than single systems: The paper challenges the common assumption that AGI will be a single monolithic AI system, instead proposing that advanced AI capabilities could first emerge through multiple specialized AI agents working together—a scenario that has received minimal safety research attention despite rapid progress in multi-agent AI systems.

Current AI safety approaches are inadequately prepared: Existing safeguards focus on controlling individual AI systems, but this framework may be insufficient if multiple less-powerful AI agents can collectively achieve general intelligence through coordination and tool-use, requiring fundamentally different safety strategies.

Coordinated AI agents pose urgent near-term risks: The paper emphasizes that advanced AI agents with communication abilities and tool-use capabilities are already being deployed rapidly, making the coordination problem an immediate concern rather than a distant theoretical issue.

Market mechanisms and sandbox environments are proposed as key safeguards: The framework suggests containing coordinated AI agents within controlled "sandbox economies" where transactions between agents are governed by market rules, auditability systems, and reputation management to prevent harmful collective behavior.

Oversight and transparency become critical at the system level: Rather than just monitoring individual agents, the approach emphasizes the need for robust auditing and oversight of agent-to-agent interactions and transactions to catch emergent risks that might not be visible when examining single systems in isolation.