Channel pruning for convolutional neural networks using l0-norm constraints

Abstract

Channel pruning can effectively reduce the size and inference time of Convolutional Neural Networks (CNNs). However, existing channel pruning methods still face several issues, including high computational costs, extensive manual intervention, difficulty in hyperparameter tuning, and challenges in directly controlling the sparsity. To address these issues, this paper proposes two channel pruning methods based on $l_0$-norm sparse optimization: the $l_0$-norm Pruner and the Automated $l_0$-norm Pruner. The $l_0$-norm Pruner formulates the channel pruning problem as a sparse optimization problem involving the $l_0$-norm and achieves a fast solution through a series of approximations and transformations. Inspired by this solution process, we devise the Zero-Norm (ZN) module, which can autonomously select output channels for each layer based on a predefined global pruning ratio. This approach incurs low computational cost and allows for precise control over the overall pruning ratio. Furthermore, to further enhance the performance of the pruned model, we have developed the Automated $l_0$-norm Pruner. This method utilizes a Bee Colony Optimization algorithm to adjust the pruning ratio, mitigating the negative impact of manually preset pruning ratios on model performance. Our experiments demonstrate that the proposed pruning methods outperform several state-of-the-art techniques. The source code for our proposed methods is available at: https://github.com/TCCofWANG/l0_prune.