A Posteriori Lipschitz Bounds for Deep Neural Networks

Abstract

SIAM Journal on Numerical Analysis, Volume 64, Issue 3, Page 779-803, June 2026.
Abstract. Deriving sharp a posteriori upper bounds of the Lipschitz constant of deep neural networks is crucial to formally guarantee the robustness of neural network based models. We analyze three existing upper bounds written for the [math] norm. We highlight the interest of working with the [math] and [math] norms and we propose two novel bounds for feed-forward fully connected neural networks, which can be extended to convolutional neural networks. Several numerical tests confirm the theoretical results, help to quantify the relationship between the presented bounds, and establish the better accuracy of the new bounds. Four numerical tests are studied: one with random matrices; two where the output has an analytical closed form; and one for convolutional neural networks trained on the MNIST dataset. The last numerical test is an application to a reference convolutional neural network. One of our bounds is optimal in the sense that it is exact for the second test uniformly with respect to the depth of the neural network and it is always better than the other ones.