Performance Optimization of Sparse Deep Neural Networks Based on GPU

Abstract

Deep neural networks are widely used in various fields. However, due to the large scale of the latest deep neural networks, the research on the sparsity of deep neural networks is constantly carried out. The implementation of the sparse deep neural network on GPU can further accelerate the computing speed of a sparse deep neural network. The performance of the GPU code of the CUDA version is far superior to the CPU codes of the Matlab version, which confirms the great superiority of the sparse deep neural network implementation on GPU. And the GPU code of the CUDA version is up x1.61 faster than the CUSPARSE version when the deep neural network has 1024 neurons and the 1920 layers.