Formalising Human-in-the-Loop: Computational Reductions, Failure Modes, and Legal-Moral Responsibility

Human oversight isn't one-size-fits-all: The paper distinguishes between different levels of human involvement in AI systems—from simple monitoring to deeply interactive collaboration—and shows that "the legal status and safety of different setups vary greatly," meaning regulators and developers need to match oversight approaches to specific AI risks rather than applying generic "human-in-the-loop" requirements.

Current laws may create false accountability: UK and EU legal frameworks often focus on HITL setups that "may not always achieve the desired ethical, legal, and sociotechnical outcomes," and the paper warns against human "scapegoating"—situations where humans are held legally responsible for AI failures that occur due to technical limitations beyond their control.

Human oversight has predictable failure modes: The paper presents a taxonomy of how HITL systems can fail, emphasizing "practical limitations of HITL setups" and showing that humans cannot always prevent AI failures even when actively involved, which has important implications for how much responsibility should legally fall on human overseers.

There's a fundamental trade-off between accountability and explainability: The research reveals "an unavoidable trade-off between attribution of legal responsibility, and technical explainability," meaning that making it easier to assign blame for AI failures may make it harder to understand why those failures occurred—a critical tension for responsible AI governance.

Technical design choices matter for legal responsibility: The paper demonstrates that "HITL setups involve many technical design decisions" that affect both safety and legal liability, suggesting that lawmakers and AI developers need to collaborate on how systems are technically designed, not just on oversight policies after the fact.