Data-Driven Pruning Algorithm Based on Result Orientation

With their outstanding performance in natural language processing tasks such as machine translation and semantic recognition, deep neural networks have attracted great attention from both academia and industry. For more complex NLP tasks, people try to add more parameters to the network, expand more layers, input larger data samples, to produce a large model to solve the complex task. However, it is not the case that the deeper the layers, the better the parameters. There is a large amount of redundant information in the parameters, which not only contributes nothing to the results, but also increases the computational burden of the model and the storage burden of the hardware. Eliminating a small amount of redundant information often has no effect on the recognition rate of the model, but slightly improves it [1], so the neural network model needs to be compressed. Existing compression methods include model pruning, parameter quantization, tensor decomposition, knowledge distmation, etc. [2] In this paper, model pruning algorithm is selected to implement a result-oriented data-driven pruning algorithm by introducing the propagation characteristics and inter-layer correlation of neural networks and automatic decision making based on Feature Map information. Finally, the effectiveness of the result - oriented data - driven pruning algorithm is proved by comparative experiments.