SafetyExplainabilityTrustworthinesscs.CL
Foveate, Attribute, and Rationalize: Towards Physically Safe and Trustworthy AI
Alex Mei, Sharon Levy, William Yang Wang
December 19, 2022
Annual Meeting of the Association for Computational Linguistics
8 citations
1 influential
Key Takeaways
-
AI systems can pose hidden physical safety risks through everyday language: The paper identifies "covertly unsafe text" as a significant problem—harmful recommendations that aren't obviously dangerous—highlighting that safety concerns extend beyond explicit threats to subtle, real-world scenarios that could injure users.
-
Explainability is essential for managing AI safety: FARM generates human-interpretable rationales showing why content is flagged as unsafe, enabling both companies and regulators to understand specific risks to different user groups rather than relying on opaque safety classifications.
-
External knowledge sources improve safety detection: By retrieving information from trustworthy sources and attributing recommendations to those sources, the framework achieved a 5.9% absolute improvement in safety classification accuracy, suggesting that grounding AI decisions in verifiable information strengthens safety outcomes.
-
Transparency about knowledge gaps matters for safety: The framework identifies missing information needed to properly assess safety in specific contexts, helping stakeholders recognize when their systems lack sufficient knowledge to make safe recommendations—a critical step toward responsible deployment.
-
Multiple stakeholders need different safety tools: The paper emphasizes that both AI developers (to manage system risks) and policymakers (to create consumer safeguards) require concrete, interpretable information about safety failures, suggesting that responsible AI requires coordination across the entire ecosystem.