Towards Systematic Qualification of Vision-Language Models for Automotive Perception Systems

Vision-Language Models pose specific safety risks in autonomous vehicles: The paper identifies two critical failure modes—hallucinations (detecting objects that aren't there) and missed detections (failing to identify objects that are present)—either of which could create dangerous driving situations.

Current AI safety testing is fragmented and insufficient: Existing verification methods typically focus on either design-time (before deployment) or run-time (during operation) testing in isolation, but the paper argues this separation is inadequate for safety-critical automotive applications that require comprehensive oversight.

A combined testing approach is necessary: The researchers propose integrating both design-time and run-time verification techniques, using structured safety-relevant annotations and synonym-based evaluation to more thoroughly assess Vision-Language Models before and during real-world use.

Standardized evaluation frameworks are needed for automotive AI: The paper demonstrates the value of testing multiple state-of-the-art models against consistent datasets (nuScenes) using a fixed ontology, suggesting that the automotive industry needs agreed-upon standards for qualifying AI systems rather than ad-hoc testing approaches.

Responsible deployment requires systematic qualification workflows: Rather than simply adopting advanced AI techniques because they exist, the paper emphasizes that automotive applications demand deliberate, multi-phase qualification processes that systematically identify and document model limitations before safety-critical deployment.