AVID: A Benchmark for Omni-Modal Audio-Visual Inconsistency Understanding via Agent-Driven Construction

Current AI systems struggle to detect when audio and video don't match, which is a basic human skill—the paper shows that state-of-the-art multimodal AI models perform poorly at identifying these cross-modal conflicts, a critical gap for trustworthy AI systems that need to recognize inconsistencies like mismatched speech or misleading combinations of sound and visuals.

The benchmark includes 11.2K videos with nearly 40,000 documented instances of audio-visual mismatches, providing researchers with a substantial dataset to test and improve AI systems' ability to spot inconsistencies across eight different types of conflicts, from deepfakes to voiceover misalignments.

Even advanced AI models fail at temporal grounding tasks, meaning they cannot accurately pinpoint when inconsistencies occur in videos—the best model achieved only 36.1% accuracy on this task, suggesting current systems cannot reliably tell users where problems exist in video content.

Specialized training significantly improves performance, with a fine-tuned model (AVID-Qwen) showing 2.8 times better results than the base model on reasoning tasks, indicating that targeted development could help AI systems better detect manipulated or misleading video content.

This work addresses a critical gap in AI safety: while existing benchmarks focus on aligned content or obvious deepfakes, this benchmark evaluates AI's ability to understand subtle, real-world inconsistencies in long-form videos—essential for preventing misinformation and ensuring AI systems can identify when content may be deceptive.