SafetyAccountabilityTransparencycs.CL
GAF-Guard: An Agentic Framework for Risk Management and Governance in Large Language Models
Seshu Tirupathi, Dhaval Salwala, Elizabeth Daly, Inge Vejsbjerg
July 1, 2025
arXiv.org
2 citations
Key Takeaways
-
One-size-fits-all monitoring is insufficient: Current automated systems for monitoring LLMs focus primarily on technical issues like hallucination, but fail to account for the specific needs of different use-cases and user preferences, highlighting a gap between generic safety measures and real-world deployment requirements.
-
Autonomous agents can enhance governance: GAF-Guard uses autonomous agents to actively identify and monitor risks rather than passively detecting problems, suggesting that AI systems themselves can be designed to take on governance responsibilities and continuously report on safety issues.
-
User-centered design is critical for responsible AI: The framework places users, their specific use-cases, and model characteristics at the center of risk management, indicating that responsible AI governance must be tailored to actual deployment contexts rather than applied uniformly across all applications.
-
Continuous monitoring enables proactive safety: By facilitating ongoing monitoring and reporting rather than one-time safety checks, the framework addresses the reality that LLM risks evolve over time and require sustained attention to maintain alignment with human values and prevent unintended negative consequences.
Abstract
As Large Language Models (LLMs) continue to be increasingly applied across various domains, their widespread adoption necessitates rigorous monitoring to prevent unintended negative consequences and ensure robustness. Furthermore, LLMs must be designed to align with human values, like preventing harmful content and ensuring responsible usage. The current automated systems and solutions for monitoring LLMs in production are primarily centered on LLM-specific concerns like hallucination etc, with little consideration given to the requirements of specific use-cases and user preferences. This paper introduces GAF-Guard, a novel agentic framework for LLM governance that places the user, the use-case, and the model itself at the center. The framework is designed to detect and monitor risks associated with the deployment of LLM based applications. The approach models autonomous agents that identify risks, activate risk detection tools, within specific use-cases and facilitate continuous monitoring and reporting to enhance AI safety, and user expectations. The code is available at https://github.com/IBM/risk-atlas-nexus-demos/tree/main/gaf-guard.