Training Provably Robust Models by Polyhedral Envelope Regularization

• Published in: IEEE transaction on neural networks and learning systems Vol. 34; no. 6; pp. 3146 - 3160
• Main Authors: Liu, Chen, Salzmann, Mathieu, Susstrunk, Sabine
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.06.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adversarial training; Benchmarks; Computational modeling; Neural networks; Predictive models; provable robustness; Recurrent neural networks; Regularization; Robustness; Robustness (mathematics); Smoothing methods; Training
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2021.3111892

Training certifiable neural networks enables us to obtain models with robustness guarantees against adversarial attacks. In this work, we introduce a framework to obtain a provable adversarial-free region in the neighborhood of the input data by a polyhedral envelope, which yields more fine-grained certified robustness than existing methods. We further introduce polyhedral envelope regularization (PER) to encourage larger adversarial-free regions and thus improve the provable robustness of the models. We demonstrate the flexibility and effectiveness of our framework on standard benchmarks; it applies to networks of different architectures and with general activation functions. Compared with state of the art, PER has negligible computational overhead; it achieves better robustness guarantees and accuracy on the clean data in various settings.