Network Slimming with Augmented Sparse Training and Optimized Pruning

Abstract

Previous works use a similar process to prune channels: train, prune, fine-tune. In this paper, we treat channel pruning as a method of network architecture search. Specifically, we limit the search space by adding some conditions on it, and after searching, we only reserve the architecture of the network and train it from scratch. We train the model with augmented sparsity to get a higher ratio of pruning. During pruning, we add a protect threshold to prevent the pruned model from being disconnection. Our process of channel pruning is as follows: train with sparsity, prune, train from scratch. we verified the effectiveness of our method on several models, including VGGNet, ResNet and DenseNet on various datasets. Otherwise, we test our method on different architectures of ResNet and analyze the results on both models.