Convolutional neural networks with coherent nanophotonic circuits

Convolutional neural networks (CNN) has significant advantages in processing image classification and was widely used in image analysis in the fields of autonomous driving, aerospace, and biomedicine. However, image classification and analysis need large matrix multiplication, which imposes many challenges to the realization of high performance and low power consumption of CNNS. Here, a photoelectric hybrid neural network (PHNN) was developed to reduce the CNN’s power consumption. The optical interference unit (OIU) composed of Mach-Zehnder interferometers (MZI) arrays, used as convolution kernel, performs multiplication and accumulation operations. The convolution kernel is split and reorganized effectively to form a new unitary matrix to reduce the number of MZIs. Simultaneously, this method can modularize the OIU, which is beneficial to field-programmable gate array (FPGA) encoding and modulation. FPGA realizes nonlinear calculation, data scheduling and storage, and phase encoding and modulation. Our PHNN has an accuracy rate of 93.3%, which reduces power consumption by 3 times of magnitude compared with traditional electronic products.