Automated Pruning of Neural Networks for Mobile Applications

Pruning is useful method to compress neural networks and further reduce the required computations and thus the inference speed. This work presents an automatic pruning workflow using an measurement based method to determine which portions of the network only contribute little to the total accuracy. Furthermore to increase the pruneability within networks containing residual blocks this work evaluates zero-padding as an useful complement to existing pruning methods. With zero-padding added to the pruning, we enable the automatic pruning process to also choose layers for pruning which would otherwise not be possible or only possible with removing additional filters which might contribute to the total accuracy. Zero-padding therefore adds the removed channels back into the original output feature map in a manner that the shapes remain identical, but the computations are saved. Using this method we achieved a speedup of up to 21% on CPU based platforms and 5-6% on GPU based execution on a MobileNetV2. The pruned network became comparable to an original network with an applied depth multiplier with only little additional retraining time.