Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling

Most AI audit tools focus narrowly on evaluation rather than accountability: The research found that while many of the 435 tools analyzed are designed to help set standards and evaluate AI systems, they "often fall short in supporting accountability"—meaning they don't adequately help practitioners identify harms, document problems, or drive real-world changes.

Significant gaps exist between what auditors need and what tools provide: Based on interviews with 35 AI audit practitioners, the study reveals that "available resources do not currently support the full scope of AI audit practitioners' needs," indicating a mismatch between the tools being built and the actual work auditors must do in practice.

Critical audit functions remain unsupported: Beyond evaluation, practitioners need tools to support "harms discovery to advocacy"—meaning the field lacks adequate infrastructure for identifying real-world harms caused by AI systems and translating audit findings into meaningful action and accountability.

The AI audit ecosystem is fragmented and incomplete: The landscape analysis of 435 tools suggests the field has not yet developed comprehensive infrastructure for AI accountability; instead, practitioners "often need to make use of various tools" to piece together their audit work, creating inefficiency and potential gaps.

Future AI accountability requires rethinking tool development: The paper calls for the field to move "beyond tools for just evaluation and towards more comprehensive infrastructure for AI accountability," suggesting that responsible AI requires systemic changes to how auditing is supported, not just better evaluation metrics.