Disclosure or Marketing? Analyzing the Efficacy of Vendor Self-reports for Vetting Public-sector AI

Self-disclosure documents like AI FactSheets serve multiple conflicting purposes simultaneously—acting as both marketing tools for vendors and evaluation documents for government buyers—which undermines their effectiveness as standalone risk-assessment instruments.

Voluntary disclosure has inherent limitations for accountability: Because vendors self-report information in public FactSheets, the structural constraints of this approach limit what gets disclosed and how thoroughly risks are assessed compared to independent evaluation.

FactSheets work best as conversation starters, not final verdicts: Rather than replacing human judgment, these documents are most effective when they facilitate ongoing dialogue and relationship-building between vendors and government agencies, helping establish shared understanding over time.

Documentation-based governance strategies need empirical validation: The study reveals there is "limited empirical evidence" about how transparency tools actually work in practice, suggesting policymakers should test whether disclosure requirements achieve their intended accountability goals before scaling them widely.

Procurement teams need complementary oversight mechanisms: FactSheets alone cannot substitute for rigorous due diligence; government agencies should use them as one input among multiple evaluation methods rather than relying on vendor self-reports as the primary basis for AI vetting decisions.