Going Shallower with MobileNets: On the Impact of Wavelet Pooling

Abstract

MobileNet is a light-weight neural network model that has facilitated harnessing the power of deep learning on mobile devices. The advances in pervasive computing and the ever-increasing interest in deep learning has resulted in a growing research attention on the enhancement of the MobileNet architecture. Beside the enhancement in convolution layers, recent literature has featured new directions for implementing the pooling layers. In this work, we propose a new model based on the MobileNet-V1 architecture, and we investigate the impact of wavelet pooling on the performance of the proposed model. While traditional neighborhood pooling can result in information loss, which negatively impacts any succeeding feature extraction, wavelet pooling allows us to utilize spectral information which is useful in most image processing tasks. On two widely adopted datasets, we evaluated the performance of the proposed model, and compared to the baseline MobileNet, we attained a 10% and a 16% increase in classification accuracy on CIFAR-10 and CIFAR-100 respectively. We also evaluated a shallow version of the proposed architecture with wavelet pooling, and we showed that it maintained the classification accuracy either higher than, or <1% less than, the deep versions of MobileNet while decreasing the number of model parameters by almost 40%.