A Discover of Class and Image Level Variance Between Different Pruning Methods on Convolutional Neural Networks

Abstract

Neural network pruning techniques have been widely used due to their little deterioration to test set accuracy while removing a great amount of weights in a network. Recent research [1] has shown that pruning impacts classification of classes and images differently even in one task. In this paper, we dive more along this line and find that different kinds of pruning methods will have different influences on classes and images, but pruning methods belonging to the same family will have a similar influence. Specifically, using iterative L1 unstructured pruning gets the least deviation for classes' accuracy from the overall accuracy and structured pruning is more likely to lead to high deviation. These findings show that choice of pruning methods can be quite nuanced and should be treated cautiously before it is used in sensitive domains.