Human-centred test and evaluation of military AI

Humans must be treated as active components of AI military systems, not afterthoughts. The paper argues that traditional AI testing frameworks are insufficient because they don't account for the human-machine interaction as an integrated system. Standards and requirements need to be redesigned to evaluate how humans and AI work together throughout the system's entire lifecycle, not just during initial development.

Continuous monitoring and evaluation is essential after deployment. The paper emphasizes that test and evaluation cannot end once a system is deployed. Military AI systems require "ongoing monitoring and evaluation" to ensure they remain safe and effective in real-world conditions, which represents a significant shift from traditional software testing practices.

Better communication between technologists and policymakers is critical for responsible AI. The paper identifies a persistent gap where operators and policymakers often don't fully understand the risks associated with AI system use. Bridging this communication gap is necessary so that research and development efforts are better informed by policy concerns and operational realities.

Human scalability poses a practical challenge to AI testing. As AI systems scale up, the ability to adequately test human-AI interactions becomes more difficult. The paper highlights that testing capacity may not keep pace with deployment scale, creating a potential oversight gap that needs to be addressed through new methodologies.

Accountability requires humans to remain responsible decision-makers. Aligned with the REAIM 2024 Blueprint, the paper stresses that humans must retain responsibility and accountability for AI system effects in military contexts, which demands robust evaluation frameworks that verify this human oversight is actually functioning.