Value-Guided Iterative Refinement and the DIQ-H Benchmark for Evaluating VLM Robustness

Real-world visual degradation poses serious risks to AI safety: The DIQ-H benchmark reveals that common real-world conditions like motion blur, sensor noise, and compression artifacts cause Vision-Language Models to make persistent errors in safety-critical applications such as robotics and autonomous systems—a vulnerability that existing benchmarks fail to capture.

Errors compound over time in continuous deployment: Unlike static testing, the benchmark demonstrates that mistakes accumulate across sequential decisions, meaning a single misinterpretation can trigger cascading failures. This "error propagation" effect is critical for evaluating systems that operate continuously rather than in isolated tasks.

Value misalignment worsens under stress: When VLMs encounter degraded visual conditions, they don't just make factual errors—they also produce outputs that conflict with intended ethical values and safety guidelines, highlighting the need to test both accuracy and alignment simultaneously.

Automated quality control can improve safety evaluation efficiency: The Value-Guided Iterative Refinement (VIR) framework improved annotation accuracy from 72.2% to 83.3% while automating the creation of ethically aligned test cases, making rigorous safety testing more scalable and cost-effective for developers.

Current VLM benchmarks are insufficient for safety-critical deployment: Existing evaluation methods focus on curated, high-quality images and don't account for real-world perturbations or value consistency over time, creating a dangerous gap between how these models are tested and how they actually perform in embodied AI applications.