AI Safety: Necessary, but insufficient and possibly problematic

AI safety is dominated by powerful actors: The paper notes that the AI safety discourse is primarily shaped by governments and corporations, rather than emerging from broader research communities focused on advancing social good, which may narrow the scope of what problems are prioritized.

"Safety" alone doesn't ensure societal benefit: The authors argue that AI safety is an "insufficient notion" for achieving broad societal good, suggesting that a system can be technically safe while still causing harm—for example, through exploitation or reinforcing structural inequalities.

AI safety can conflict with transparency: The paper identifies an "uneasy relationship" between AI safety concepts and transparency, implying that some safety measures may actually reduce public understanding and accountability of AI systems rather than enhance it.

Safety framing may legitimize harmful AI: A key concern raised is that focusing narrowly on "safety" could provide "a veneer of safety" to exploitative or structurally harmful AI systems, potentially allowing problematic applications to proceed if they meet technical safety standards.

AI safety is already shaping regulation in potentially problematic ways: The authors warn that AI safety concepts have already influenced regulatory efforts, suggesting these influences may not be moving in desirable directions for society broadly.