SafetyBiasEthicscs.CL
Granite Guardian
Inkit Padhi, Manish Nagireddy, Giandomenico Cornacchia, Subhajit Chaudhury, Tejaswini Pedapati, Pierre Dognin, Keerthiram Murugesan, Erik Miehling, Martín Santillán Cooper, Kieran Fraser, Giulio Zizzo, Muhammad Zaid Hameed, Mark Purcell, Michael Desmond, Qian Pan, Zahra Ashktorab, Inge Vejsbjerg, Elizabeth M. Daly, Michael Hind, Werner Geyer, Ambrish Rawat, Kush R. Varshney, Prasanna Sattigeri
December 10, 2024
arXiv.org
28 citations
4 influential
Key Takeaways
-
Comprehensive risk coverage: Granite Guardian detects seven major risk categories including social bias, profanity, violence, sexual content, unethical behavior, jailbreaking attempts, and hallucination-related issues—addressing gaps that traditional safety models miss, particularly around jailbreaks and retrieval-augmented generation (RAG) problems.
-
Works with any LLM: Rather than being tied to a specific language model, these safeguard models function as an add-on layer that can protect users regardless of which LLM they're using, making responsible AI practices more universally accessible.
-
Strong performance metrics: The models achieved AUC scores of 0.871 for harmful content detection and 0.854 for RAG-hallucination detection, positioning them as the most generalizable risk detection models currently available according to the paper's benchmarking.
-
Trained on diverse, real-world data: The models were developed using both human annotations from multiple sources and synthetic data, helping them recognize nuanced risks that reflect actual usage patterns rather than theoretical scenarios.
-
Open-source for community benefit: By releasing Granite Guardian as open-source software, the developers aim to democratize responsible AI safeguards, enabling broader adoption of safety practices across organizations rather than concentrating these tools with a single company.