Are Safety Guarantees in Neural Networks Safe? How to Compute Trustworthy Robustness Certifications

Adversarial examples pose a real safety challenge: Neural networks can be fooled by slightly altered inputs that humans would recognize correctly, making it crucial to understand and measure how robust AI systems actually are to such attacks.

Current safety certifications may be misleading: The paper reveals that most existing robustness certifications prioritize size (volume) over other measures, but computing truly optimal certifications this way is mathematically impossible within reasonable timeframes—meaning organizations claiming to have "certified safe" AI systems may be using incomplete or unreliable measures.

A better measurement approach exists: The researchers propose using the "apothem measure" (essentially the distance from the center to the nearest edge of a safety zone) as a more computationally feasible alternative, and demonstrate their ParallelepipedoNN system achieves at least twice the improvement in certification quality compared to existing methods on standard benchmarks.

Verification requires systematic checking: The paper emphasizes that trustworthy robustness certifications require multiple calls to neural network verifiers—there's no shortcut to genuinely understanding whether an AI system will behave safely when faced with slightly corrupted or adversarial inputs.

Transparency about limitations matters for responsible AI: By proving that volume-optimal certifications are computationally intractable, this work highlights the importance of being honest about what safety guarantees can and cannot deliver, rather than overstating the robustness of AI systems.