Misalignment or misuse? The AGI alignment tradeoff

Both misalignment and misuse pose genuine catastrophic risks: The paper argues that while misaligned AGI (AI systems that don't pursue human goals) is dangerous, aligned AGI that follows human instructions could be equally catastrophic if misused by bad actors—creating a fundamental tension in AI safety approaches.

Current alignment techniques may inadvertently increase misuse risks: Many existing methods to make AI systems more aligned and controllable could paradoxically make them more useful for harmful purposes, suggesting that standard alignment approaches alone are insufficient for ensuring safe AI development.

Technical solutions alone cannot solve the problem: The paper emphasizes that governance, robustness, and AI control methods are "essential" alongside technical alignment work, indicating that responsible AI requires social and institutional safeguards, not just better algorithms.

There is theoretical room for alignment without increasing misuse risk: While acknowledging the tradeoff exists, the authors argue that in principle, alignment approaches could be designed that don't make catastrophic misuse more likely—though achieving this requires careful consideration of how techniques are implemented.

Social context matters critically: The paper stresses that AI risks depend heavily on the social environment in which systems are deployed, suggesting that responsible AI development must account for real-world governance structures, institutional incentives, and potential bad actors.