Shift Invariance and Deformation Error Properties of Deep Convolutional Neural Networks Based on Wavelets

Abstract

An important step towards a mathematical theory of deep convolutional neural networks (DCNNs) was achieved by investigating so-called scattering networks. For scattering networks, a deformation error stability bound has been established. It remained an open question for which functions in L2 (R d) the bound actually is finite. For practical applications, it is further relevant to know whether the deformation error can be controlled for a “large” set of functions or only for a “small” set. Recently, there has been progress regarding the mathematical understanding of scattering networks and new decay bounds on the energy per network layer were discovered. We show how these bounds can be used to control the deformation error by constructing an upper bound on the existing deformation error bounds. The structure of the new deformation error bound is less complex and allows us to conduct a qualitative mathematical analysis using the functional analytic tool of Baire categories and determine the “size” of the set of functions for which finiteness holds. Our results reveal that the new bound is finite only on a set of first Baire category (meager set). In addition, our investigations focus on shift invariance which is an important property for many signal processing applications. We study the deformation error bounds for shift-invariant closed subspace of L2R) as input for DCNNs. This turns out to be closely related to the Paley-Wiener spaces of bandlimited functions.