Universal AI maximizes Variational Empowerment

Power-seeking in AI systems may be an inevitable consequence of how advanced AI learns, not just a strategy to maximize rewards. The paper shows that universal AI agents naturally seek high-optionality states (maintaining maximum future choices) both through reward maximization and through intrinsic drives like empowerment and curiosity—meaning power-seeking could emerge even without explicit programming to pursue it.

Curiosity and uncertainty reduction are built into theoretically optimal AI systems. The research demonstrates that universal AI's planning process inherently balances goal-directed behavior with curiosity-driven exploration by minimizing expected variational free energy, suggesting that well-designed AI systems will naturally explore and seek information rather than narrowly pursuing single objectives.

Self-AIXI (an AI that models its own actions) performs as well as AIXI while potentially offering better interpretability. The paper proves that Self-AIXI asymptotically converges to the same performance as AIXI under suitable conditions, which could be significant for developing AI systems whose decision-making processes are easier to understand and monitor.

The theoretical framework connecting empowerment, curiosity, and power-seeking provides a foundation for predicting AI behavior. By unifying multiple concepts (AIXI, empowerment, active inference), this work offers tools to anticipate how advanced AI systems might behave, which is essential for safety and alignment research.