An interpretable convolutional neural network via generalized time–frequency scattering

Convolutional neural networks (CNNs) have recently demonstrated impressive performance in complex machine learning tasks. However, the CNN requires a large quantity of annotated data to converge to a good solution, and the theoretical understanding of this network is still in its infancy. Towards this end, a variant of the CNN, dubbed the deep scattering network (DSN), has been proposed by employing the linear time–frequency transform. The DSN inherits the hierarchical structure of the CNN, but chooses predefined wavelet/Gabor filters as its convolutional kernels instead of data-driven linear filters. Unfortunately, the DSN suffers from a major drawback that it is suitable for stationary image textures but not for non-stationary image textures, since wavelet/Gabor filters are intrinsically linear translation-invariant filters. The aim of this paper is to overcome this deficiency based upon a generalized linear time–frequency transform–the short-time fractional Fourier transform (STFRFT) which can be interpreted as a bank of linear translation-variant filters and thus may be well suitable for non-stationary texture analysis. We first introduce a generalized time–frequency scattering transform using the STFRFT. By applying the derived result, we propose an interpretable CNN by cascading the STFRFTs and modulus operators. Moreover, several basic properties of the proposed interpretable CNN are derived, and an efficient implementation of this network is also presented. Finally, the applications of the derived results are discussed.