Maximum redundancy pruning for network compression

Abstract

Filter pruning has become one of the most powerful methods for model compression in recent years. However, existing pruning methods often rely on predefined layer-wise pruning ratios or computationally expensive search processes, leading to suboptimal architectures and high computational overhead. To address these limitations, we propose a novel pruning method, termed Maximum Redundancy Pruning (MRP), which consists of Redundancy Measurement by Community Detection (RMCD) and Structural Redundancy Pruning (SRP). We first demonstrate a Role-Information (RI) hypothesis based on the link between social networks and convolutional neural networks through empirical study. Based on that, RMCD is proposed to obtain the level of redundancy for each layer, enabling adaptive pruning without predefined layer-wise ratios. In addition, we introduce SRP to obtain a sub-network with the optimal architecture according to the redundancy of each layer obtained by RMCD. Specifically, we recalculate the redundancy of each layer at each iteration and then remove the most replaceable filters in the most redundant layer until a target compression ratio is achieved. This approach automatically determines the optimal layer-wise pruning ratios, avoiding the limitations of uniform pruning or expensive architecture search. We show that our proposed MRP method can reduce the model size for ResNet-110 by up to 52.4% and FLOPs by up to 50.3% on CIFAR-10 while actually improving the original accuracy by 1.04% after retraining the networks.