Design and evaluation of AI copilots -- case studies of retail copilot templates

Responsible AI guardrails are a core architectural component, not an afterthought. The paper identifies guardrails as a key technical element alongside language models and plugins, emphasizing that safety considerations must be built into copilot design from the ground up rather than added later.

Human-AI decision loops require systematic testing and evaluation. The paper advocates for an "end-to-end human-AI decision loop framework" to measure quality and safety, suggesting that responsible AI requires ongoing assessment of how humans and AI systems interact in real business contexts, not just isolated AI performance metrics.

Orchestration and system prompts significantly shape copilot behavior. By highlighting orchestration and system prompts as critical technical components, the paper indicates that how a copilot is instructed and how its different capabilities are coordinated directly influence whether it produces desired outcomes and avoids unintended consequences.

Managing unintended consequences requires principled evaluation practices. The paper explicitly frames testing and evaluation as "a principled way to promote desired outcomes and manage unintended consequences," suggesting that responsible AI deployment demands structured approaches to identifying and mitigating harms before systems reach users.

Domain-specific design matters for responsible AI. The use of retail copilot templates as a case study suggests that responsible AI principles must be adapted to specific business contexts and use cases, rather than applying generic solutions across different domains.