Who Does What in AI Auditing? Designing Human-AI Collaboration for Auditing Generative AI

AI assistance in auditing can be a double-edged sword: While AI agents helped auditors find more problems and explore broader attack scenarios against AI systems, the study found that auditors became overly reliant on AI-generated assessments and reports, potentially reducing critical human judgment—the very thing auditing is meant to preserve.

Human oversight remains essential for contextual judgment: The research shows that certain auditing tasks—particularly exploration, assessment, reporting, and review—require different levels of human involvement. The framework (HAAC) emphasizes that humans must retain decision-making authority in areas requiring contextual understanding rather than delegating these entirely to AI agents.

Auditors need transparency into how audits are conducted: Interviews with 71 auditors and Responsible AI practitioners revealed that effective auditing requires visibility into what coverage was achieved, reproducible documentation of how problems were found, and crucially, evaluation of the auditing AI agents themselves—not just the systems being audited.

AI auditing agents themselves need to be audited: A key finding is that organizations cannot simply trust AI agents to conduct audits without scrutiny. The paper emphasizes that auditing workflows must include mechanisms to evaluate whether the AI agents performing the audit are themselves working reliably and fairly.

Current AI auditing practices lack standardized collaboration models: Before this research, little was known about how to effectively divide auditing work between humans and AI without losing human judgment, suggesting that responsible AI practices need clearer guidelines for human-AI collaboration in oversight roles.