SafetyEthicsAccountabilitycs.CY
What Is AI Safety? What Do We Want It to Be?
Jacqueline Harding, Cameron Domenico Kirk-Giannini
May 5, 2025
Philosophical Studies
4 citations
Key Takeaways
-
AI safety should address all harms from AI systems, not just catastrophic risks. The paper argues against narrowing AI safety to focus primarily on existential or catastrophic risks from future systems, advocating instead for a broader conception that treats all efforts to prevent AI harms as equally valid based on their merits.
-
Current AI harms deserve equal attention to speculative future risks. The paper emphasizes that work on present-day issues like bias, misinformation, and privacy should be considered central to AI safety rather than marginal, suggesting these concrete harms affecting people today warrant the same research priority as theoretical catastrophic scenarios.
-
AI safety is broader than engineering alone. While some researchers frame AI safety as a branch of safety engineering, the paper argues this framing is too narrow and misses important dimensions of how we should think about preventing AI harms across different contexts and applications.
-
How we define AI safety matters for research priorities and funding. The paper uses "conceptual engineering" to argue that adopting the broader "Safety Conception" would better guide which research projects get pursued and supported, potentially redirecting attention toward addressing harms that currently affect real people.
Abstract
The field of AI safety seeks to prevent or reduce the harms caused by AI systems. A simple and appealing account of what is distinctive of AI safety as a field holds that this feature is constitutive: a research project falls within the purview of AI safety just in case it aims to prevent or reduce the harms caused by AI systems. Call this appealingly simple account The Safety Conception of AI safety. Despite its simplicity and appeal, we argue that The Safety Conception is in tension with at least two trends in the ways AI safety researchers and organizations think and talk about AI safety: first, a tendency to characterize the goal of AI safety research in terms of catastrophic risks from future systems; second, the increasingly popular idea that AI safety can be thought of as a branch of safety engineering. Adopting the methodology of conceptual engineering, we argue that these trends are unfortunate: when we consider what concept of AI safety it would be best to have, there are compelling reasons to think that The Safety Conception is the answer. Descriptively, The Safety Conception allows us to see how work on topics that have historically been treated as central to the field of AI safety is continuous with work on topics that have historically been treated as more marginal, like bias, misinformation, and privacy. Normatively, taking The Safety Conception seriously means approaching all efforts to prevent or mitigate harms from AI systems based on their merits rather than drawing arbitrary distinctions between them.