Facilitating Trustworthy Human-Agent Collaboration in LLM-based Multi-Agent System oriented Software Engineering

Clear task allocation reduces risks: The paper proposes a RACI-based framework (Responsible, Accountable, Consulted, Informed) to strategically divide work between humans and AI agents in software engineering, helping ensure that critical decisions remain under human oversight rather than being fully automated.

Accountability is essential for AI systems: By establishing clear roles and responsibilities through the RACI framework, the research addresses a major gap in current LLM-based multi-agent systems—ensuring someone (human or system) is accountable for each decision, which is crucial for responsible AI deployment.

Multi-agent systems need human-in-the-loop safeguards: The framework explicitly aims to "mitigate potential risks associated with LLM-driven automation," recognizing that autonomous AI systems in software engineering require human collaboration checkpoints rather than full autonomy.

Alignment with trustworthy AI principles: The proposed approach is designed to align with established Trustworthy AI guidelines, suggesting that responsible human-agent collaboration is a practical pathway to implementing broader AI governance standards in real-world applications.

Empirical validation still needed: The authors acknowledge this is "work in progress" and plan future empirical validation, meaning the framework's real-world effectiveness in software engineering contexts has not yet been tested or proven.