Matched Wavelet Pooling for Efficient Light-Weight CNN Network Architectures with Application to Image Classification

This research addresses the challenge of evaluating the capability of an emerging pooling method, namely, wavelet pooling, to increase the data efficiency in deep neural networks. Specifically, we focus on light-weight network architectures. The volume of training data required to achieve a particular performance level, e.g., recognition accuracy, is what we refer to as data efficiency. Recently, wavelet pooling has been attracting an increasing research attention due to its capacity to overcome the spatial information loss that is inherent in traditional pooling, by capitalizing on localized spectral information in the wavelet domain. We proposed a new wavelet pooling approach recently which has shown significant promise specifically on MobileNets. This approach chooses which wavelet band(s) to include during training by matching the input images (to be recognized) to specific wavelet sub-bands, hence the name-Matched Wavelet Pooling (MWP). How does the performance of MobileNets differ when MWP is adopted compared to the performance of baseline MobileNets and MobileNets with non-matched wavelet pooling? The principal contribution of this research is addressing this research question. We hypothesize that MWP when used on MobileNets requires a smaller volume of training, than baseline MobileNets and non-matched wavelet pooling, to achieve the same recognition accuracy. On two popular benchmarks, namely, CINIC-10 and STL-10 datasets, we report consistent data savings (achieved by MWP) that approaches 30% compared to the baseline MobileNet model and the non-matched wavelet pooling while achieving higher recognition accuracy.