Market Incentives for AI Safety Investment

Downstream firms over-invest in safety due to competition: The paper finds that competing downstream AI companies actually invest more in safety measures than would maximize industry profits, suggesting market competition can drive safety spending beyond what's economically optimal for the sector as a whole.

Upstream providers systematically under-invest in safety: Large language model providers (the "upstream" firms supplying base models) tend to underinvest in safety because they don't capture the full financial benefits of their safety improvements—the value flows to downstream companies using their models, creating a misalignment between who pays and who benefits.

The AI supply chain structure matters for safety outcomes: The paper demonstrates that how AI systems are organized—with some companies building foundational models and others building applications on top—fundamentally shapes safety incentives in ways that neither market competition nor profit-seeking alone can fully resolve.

Regulation can reshape safety incentives: Different regulatory approaches affect how much each type of company invests in safety and their profitability, suggesting policymakers have tools to influence safety investment levels beyond relying on market forces alone.

Profit maximization doesn't align with societal welfare on safety: The market-driven level of safety investment differs from what would maximize overall societal welfare, indicating that voluntary corporate safety spending may be insufficient to address risks like toxicity, malicious use, and misinformation.