AccountabilitySafetySecuritycs.AI
Towards Responsible Generative AI: A Reference Architecture for Designing Foundation Model based Agents
Qinghua Lu, Liming Zhu, Xiwei Xu, Zhenchang Xing, Stefan Harrer, Jon Whittle
November 22, 2023
2024 IEEE 21st International Conference on Software Architecture Companion (ICSA-C)
33 citations
1 influential
Key Takeaways
-
Foundation model-based agents can autonomously break down complex goals into manageable tasks, but this autonomy introduces significant risks that current architectural designs don't systematically address—the paper identifies a gap in how these systems are engineered for responsible deployment.
-
Security and accountability are critical software quality concerns for AI agents that have not been adequately incorporated into agent design practices; the reference architecture proposed in this paper explicitly integrates these responsible AI considerations into the foundational design patterns.
-
A pattern-oriented reference architecture can guide responsible agent design by providing systematic guidance for developers, as demonstrated through the paper's evaluation mapping the architecture to two real-world agent implementations.
-
The autonomous planning and execution capabilities of foundation model-based agents offer significant benefits, but require careful architectural consideration to ensure these systems operate safely and can be held accountable for their actions—suggesting that responsible design must happen at the architectural level, not as an afterthought.
Abstract
Foundation models, such as large language models (LLMs), have been widely recognised as transformative AI technologies due to their capabilities to understand and generate content, including plans with reasoning capabilities. Foundation model based agents derive their autonomy from the capabilities of foundation models, which enable them to autonomously break down a given goal into a set of manageable tasks and orchestrate task execution to meet the goal. Despite the huge efforts put into building foundation model based agents, the architecture design of the agents has not yet been systematically explored. Also, while there are significant benefits of using agents for planning and execution, there are serious considerations regarding responsible AI related software quality attributes, such as security and accountability. Therefore, this paper presents a pattern-oriented reference architecture that serves as guidance when designing foundation model based agents. We evaluate the completeness and utility of the proposed reference architecture by mapping it to the architecture of two real-world agents.