Measuring What Matters: A Framework for Evaluating Safety Risks in Real-World LLM Applications

Safety testing must move beyond the base model: The paper demonstrates that evaluating only the foundation LLM is insufficient, as real-world applications introduce additional safety risks through system prompts, retrieval systems, and guardrails that can significantly alter the model's actual behavior and safety profile.

Organizations need customized safety frameworks: Rather than applying one-size-fits-all safety standards, the paper advocates for developing tailored safety risk taxonomies specific to each organization's use cases, recognizing that different applications face different types of risks.

Application-level safety requires systematic evaluation practices: The framework provides concrete practices for testing safety at the deployment stage, bridging the gap between theoretical AI safety research and the practical operational challenges organizations face when rolling out LLM applications.

Real-world deployment reveals safety issues that lab testing may miss: The paper's validation through actual organizational deployments across multiple use cases suggests that safety risks only become apparent when systems are tested in realistic operational contexts, not just in controlled research settings.

Scaling safe LLM deployment requires actionable guidance: The paper emphasizes that organizations need practical, implementable approaches to safety testing—not just theoretical frameworks—to responsibly scale their use of LLMs while managing risks effectively.