Towards Assurance of LLM Adversarial Robustness using Ontology-Driven Argumentation

LLMs are vulnerable to adversarial attacks that require systematic defense strategies: The paper identifies that large language models face security challenges from adversarial attacks, necessitating a combination of adversarial training (teaching models to resist attacks) and guardrails (safety constraints) to protect them.

Current approaches to managing LLM safety lack transparency and organization: The researchers highlight that existing knowledge about attacks and defenses is "implicit and heterogeneous," making it difficult for organizations to systematically track and assure that their models are actually robust—a critical gap for responsible AI deployment.

Formal argumentation and ontologies can create understandable safety documentation: The paper proposes using structured logical frameworks and knowledge organization systems to create "human-readable assurance cases" that explain why an LLM is safe, alongside machine-readable versions that can be automatically verified—improving transparency for both technical teams and auditors.

This approach serves multiple stakeholder needs in AI governance: By targeting engineers, data scientists, users, and auditors, the framework acknowledges that responsible AI requires different types of evidence and explanations for different audiences, from those building systems to those overseeing them.

Practical application extends beyond text to code translation tasks: The demonstration with both English language and code translation examples suggests this assurance method could apply across different LLM use cases, supporting broader responsible AI practices.