Bayesian Filter Pruning for Deep Convolutional Neural Network Compression

Abstract

Network pruning has been demonstrated as a feasible approach in reducing model complexity and accelerating the process of inference, which make it possible to deploy deep neural network in resource-limited devices. Many previous works on network pruning consider the magnitude of parameters or other intrinsic properties in point-estimates based network as the criterion of module selection, which are incapable of estimating uncertainty of parameters. In this paper, we propose a novel Bayesian filter pruning method, which leverages the advantage of Bayesian Deep Learning (BDL), by exploring the properties of distribution in weight. The proposed method removes redundant filters from a Bayesian network by a criterion of the proposed Signal to Noise Ratio (SNR) that combines properties of importance with uncertainty of filters. Experimental results on two benchmark datasets show the efficiency of our method in maintaining balance between compression and acceleration.