A Chain Is Only as Strong as Its Weakest Link: A Scoping Review of System Integration Audits in AI

Most AI audits focus too narrowly on individual models rather than how systems work together: The review found that "model-centric audits are insufficient to address risks arising from interactions among system components and deployment environments," meaning current practices miss problems that emerge when AI components interact with each other and real-world settings.

System integration auditing is still underdeveloped in AI despite being standard practice in safety-critical fields: While aerospace and other industries have long used system integration audits, the review identified only 58 relevant AI audits out of 4,259 documents examined, indicating this approach remains "an emerging yet still fragmented form of AI auditing" with "few existing measures" targeting integration-specific risks.

Three critical areas need evaluation: how AI components interact with each other, how systems interact with their environments, and how multiple AI systems work together: The research identifies these "three sites" of integration (inter-component, system-environment, and multi-system) as essential areas for auditors to assess, each serving functions like "risk exploration, risk determination, coordination, and procedural regularity."

Access to information and resources is a major barrier to conducting thorough AI system audits: The review found that "access to necessary information and resources significantly influences audit design," suggesting that transparency and cooperation from AI developers are critical enablers for effective auditing.

Auditors should evaluate specific integration qualities like compatibility, completeness, and oversight: These "qualities specific to system integration" differ from traditional model evaluations and are essential for catching "failures across components."