Generalizable Robustness Testing of DNN-Based Robotic Navigation Systems via XAI-Guided Search

AI systems controlling robots need rigorous real-world testing: The research demonstrates that vulnerabilities found only in computer simulations don't always transfer to physical robots (67% recall in sim-to-real evaluation), highlighting the critical gap between testing in controlled environments and actual deployment safety.

Small, invisible changes to sensor inputs can cause dangerous failures: The study shows that sparse perturbations—minimal alterations to what a robot's camera sees—can trigger unsafe navigation behavior, raising concerns about how robust AI-controlled systems are to real-world variations like lighting changes or camera artifacts.

Explainability tools can make AI testing more efficient: By using explainability methods (Integrated Gradients) to guide the search for failure cases, researchers improved test success rates from 54% to 70%, suggesting that understanding why an AI makes decisions helps identify vulnerabilities faster than random testing.

Testing multiple scenarios together is more effective than testing one at a time: Multi-image optimization improved failure detection from 50% to 57.5%, indicating that robustness tests need to account for how AI systems behave across diverse real-world conditions, not just isolated cases.

Responsible deployment requires validation on actual hardware: The significant correlation (0.617) between simulated and physical robot failures, while positive, still leaves substantial unexplained variance, emphasizing that safety-critical robotic systems should undergo physical testing before real-world deployment.