A High-Performance SCNN Accelerator Using Parallel Sparsity Detection and Index-Oriented Computation Workflow

Abstract

The customization of accelerators for sparse convolutional neural networks (SCNNs) has been shown to significantly enhance the computational efficiency of CNNs. However, while processing the widely existing irregularly distributed sparsity in filters and feature maps, serial sparsity detection (SSD) methods and small-capacity computation arrays are always applied in current works. As a result, it is difficult to fully translate the exploitation of sparsity into hardware performance improvement. Therefore, in this article, first, a novel parallel sparsity detection (PSD) scheme is proposed and hardware-implemented to efficiently extract the valid weights and activations. In addition, an index-oriented computation workflow for parallel sparse convolution is also proposed to eliminate the output index diversity during sparse convolutions. With the assistance of the above sparsity detection scheme and computation workflow, a large-scale two-side SCNN accelerator is designed and implemented on the Xilinx VCU118 platform, achieving a runtime frequency of 300 MHz. The evaluation results indicate that this work can achieve 1284.43/1105.31 GOPS performance while deploying VGG16/ResNet-50. Compared to the previous dense-/sparse-based works, this work can achieve a performance enhancement ranging from $1.284\times $ to $12.266\times $ and a DSP efficiency improvement from $1.718\times $ to $6.131\times $ . These results highlight the superior ability to translate sparsity exploitation into performance gains.