Concept-Guided LLM Agents for Human-AI Safety Codesign

LLMs alone are insufficient for safety-critical applications: The paper emphasizes that using Large Language Models without additional safeguards cannot meet the quality demands required for safety engineering, where failures could cause harm to people. More sophisticated approaches combining multiple techniques are necessary.

Human oversight remains essential: The research stresses that "humans must understand and take responsibility for the suggestions provided by generative AI to ensure system safety," positioning human-AI codesign rather than full automation as the appropriate model for safety-critical decisions.

Structured reasoning improves AI safety analysis: The proposed method uses "a cascade of micro-decisions that help preserve structured information" through prompt engineering, heuristic reasoning, and retrieval-augmented generation, demonstrating that guiding LLM reasoning through predefined safety concepts produces more reliable outputs than unstructured prompting.

Intermediate representations bridge human understanding and AI capabilities: The paper introduces "graph verbalization" as an intermediate representation between system models and LLMs, suggesting that translating complex technical information into formats that both humans and AI can interpret helps maintain transparency and accountability in safety analysis.

Domain-specific customization is necessary: Rather than using generic LLMs, the research develops a "customized LLM agent" tailored to safety analysis tasks, indicating that responsible AI deployment in high-stakes domains requires adaptation to specific safety concepts and requirements rather than one-size-fits-all solutions.