VLDBench Evaluating Multimodal Disinformation with Regulatory Alignment

Multimodal disinformation is a growing blind spot: Most AI safety tools focus on detecting false information in single formats (text or images alone), but real-world disinformation increasingly combines manipulated text and images together—such as fake news articles with doctored photos—which existing systems struggle to catch.

Visual information significantly improves detection: When AI models were given both text and images to analyze, their accuracy in detecting disinformation improved by 5 to 35 percentage points compared to text-only analysis, demonstrating that a complete picture is essential for catching sophisticated fakes.

High-quality benchmarks require substantial human expertise: Creating VLDBench required 22 domain experts to spend over 500 hours manually reviewing and labeling approximately 62,000 text-image pairs, highlighting that reliable AI safety tools depend on careful human oversight rather than automation alone.

Regulatory alignment matters for AI safety research: The benchmark was deliberately designed to align with AI governance frameworks like the MIT AI Risk Repository, suggesting that responsible AI development requires coordination between researchers and policymakers to ensure safety tools address real regulatory concerns.

The scale of the problem demands better tools: With AI tools making it increasingly easy to generate and spread synthetic content, VLDBench provides the first large-scale resource (covering 13 disinformation categories from 58 news outlets) specifically designed to help develop and test systems that can detect intentional multimodal manipulation at scale.