Pruning Networks Using Filters Similarity Stability

Abstract

Current filter pruning methods rely too much on pretrained weights and have many super parameters, resulting in obvious performance degradation and too long parameters adjustment time. In our research, we found that the cosine similarity distribution between filters can achieve stable in a few epochs during training. Therefore, a cluster pruning method named ECP(Early Cluster Pruning) based on the cosine similarity between filters in the early stage of training is proposed to compress the deep neural networks. First, in the early stage of training, the filters were clustered with a gradually increasing threshold, and then the reserved filters were selected randomly in each cluster. The pruned models could be obtained with only a few super parameters and a single training progress, leading to an obvious reduction in algorithmic complexity and large savings in training time. The experimental results on CIFAR-10 and CIFAR-100 datasets show that ECP method outperforms recent pruning methods in terms of model accuracy maintenance, training time, and model compression rate.