On-chip non-volatile all-optical residual neural network accelerator

Abstract

Optical neural networks (ONNs) hold substantial potential in artificial intelligence, promising faster processing speed and reduced energy consumption compared to traditional electronic neural networks, by implementing matrix operations with optical computations. Current ONN architectures predominantly rely on single- or multi-channel convolutions to accelerate computing operations. However, high-performance neural networks, such as ResNet-50, SSD, and Transformer, employ residual convolutions for deep feature extraction instead of single- or multi-channel convolutions. To make ONNs widely functioned in deep neural networks, we propose the Non-Volatile All-Optical Residual Convolution Accelerator (NARCA), based on phase change material (PCM), which weight update energy consumption is only 9.8 μW, much smaller than that of thermal-optical unit with about 10 mW. The NARCA remain high convolution precision, and the experimental results show that the NARCA-based architecture outperforms the conventional optical convolution architecture across different neural-network tasks. Moreover, we demonstrate 128 GHz high-speed optical residual convolution, which greatly improves the residual convolution operation speed compared with the electrical architecture, with a relative root mean square error (RMSE) of less than 0.125.