A Clustering Pruning Method Based on Multidimensional Channel Information

Abstract

Pruning convolutional neural networks offers a promising solution to mitigate the computational complexity challenges encountered during application deployment. However, prevalent pruning techniques primarily concentrate on model parameters or feature mapping analysis to devise static pruning strategies, often overlooking the underlying feature extraction capacity of convolutional kernels. To address this, the study first quantitatively expresses the feature extraction capability of convolutional channels from three aspects: global features, distribution metrics, and directional metrics. It explores the multi-dimensional information of the channels, calculates the overall expectation, variance, and cosine distance from the unit vector as the quantitative results of the channels. Subsequently, a clustering algorithm is employed to categorize the multidimensional information. This approach ensures that convolutional channels grouped within each cluster possess similar feature extraction capabilities. An enhanced differential evolutionary algorithm is utilized to optimize the number of clustering centers across all convolutional layers, ensuring optimal grouping. The final step involves achieving channel sparsification through the calculation of crowding distances for each sample within its designated cluster. This preserves a diverse subset of channels that are critical for maintaining model accuracy. Extensive empirical evaluations conducted on three benchmark image classification datasets demonstrate the efficacy of this method. For instance, on the ImageNet dataset, the ResNet-50 model experiences a substantial reduction in FLOPs by 58.43% while incurring a minimal decrease in TOP-1 accuracy of only 1.15%.