A Systolic Computing-in-Memory Array based Accelerator with Predictive Early Activation for Spatiotemporal Convolutions

Residual (2+1)-dimensional convolution neural network (R(2+1)D CNN) has achieved great success in video recognition due to the spatiotemporal convolution structure. However, R(2+1)D CNN incurs large energy and latency overhead because of intensive computation and frequent memory access. To solve the issues, we propose a digital SRAM-CIM based accelerator with two key features: (1) Systolic CIM array to efficiently match massive computations in regular architecture; (2) Digtal CIM circuit design with output sparsity predicition to avoid redundant computations. The proposed design is implemented in 28nm technology and achieves an energy efficiency of 21.87 TOPS/W at 200 MHz and 0.9 V supply voltage.