Reliability Evaluation of Pruned Neural Networks against Errors on Parameters

Abstract

Convolutional Neural Networks (CNNs) are widely used in image classification tasks. To fit the application of CNNs on resource-limited embedded systems, pruning is a popular technique to reduce the complexity of the network. In this paper, the robustness of the pruned network against errors on the network parameters is examined with VGG16 as a case study. The effects of errors on the weights, bias, and batch normalization (BN) parameters are evaluated for the network with different pruning rates based on error injection experiments. The results show that in general networks with more weights pruned are more robust for a given error rate. The effect of multiple errors on bias or BN parameters is almost the same for the networks with different pruning rates that are lower than 90%. Further experiments are performed to explain the bimodal phenomenon of the network performance with errors on the parameters, to find that only errors on 6% of the parameter bits will cause large degradation of the neural network performance.