SysCIM: A Heterogeneous Chip Architecture for High-Efficiency CNN Training at Edge

Abstract

Neural network training is notoriously computationally intensive and time-consuming. Quantization technology is promising to improve training efficiency by using lower data bitwidths to reduce storage and computing requirements. Currently, state-of-the-art quantization training algorithms have a negligible loss of accuracy, which requires dedicated quantization circuits for dynamic quantization of large amounts of data. In addition, the matrix transposition problem during neural network training gradually becomes a challenge as the network size increases. To address this problem, we propose a quantized training architecture which is a heterogeneous architecture consisting of a computing-in-memory (CIM) macro and a systolic array. First, the CIM macro realizes efficient transpose matrix multiplication through flexible data path control, which handles the need for transpose operation of the weight matrix in neural network training. Second, the systolic array utilizes two different data flows in the forward (FW) and backward (BW) propagation for the transpose matrix multiplication of the activation matrix in neural network training and provides higher computational throughput. Then, we design efficient dedicated quantization circuits for quantization algorithms to support efficient quantization training. Experimental results show that the area and power consumption of the two specialized quantization circuits are reduced by a factor of 1.35 and 5.4, on average, compared to floating-point computing circuits. The architecture achieves 4.05 tera operations per second per wat (TOPS/W) energy efficiency @ INT8 convolutional neural network (CNN) training at the 28-nm process. Compared to a state of the art (SOTA) quantization training architecture, SysCIM shows $1.8\times $ energy efficiency.