Optical‐Vortex‐Convolution‐Enhanced Neural Network

The optical‐electronic hybrid neural network combines the high bandwidth and processing speed of optical computing with the flexibility of electronic computing, offering a range of advantages. However, how to construct the desired optical operation layer in a practical system to realize scalable image convolution and recognition, especially for infrared images, has not been fully explored, which is urgent for diverse fields. Here, an optical‐electronic hybrid convolutional neural network (CNN) is constructed for infrared image classification and recognition. The key component is a nonlinear multi‐kernel vortex filter, namely, the optical convolutional layer, which is implemented by effectively imprinting a multiplexed vortex grating onto a potassium titanyl phosphate (KTP) crystal. The integration of this nonlinear filter with sum frequency generation (SFG) in the Fourier domain enables the extraction of visible features from infrared input images. By feeding the extracted features into an electronic fully connected layer, classification accuracies of 99.08% and 88.6% is achieved for handwritten digit recognition and fashion image recognition, respectively. These results are comparable to the performance of conventional CNNs. The findings hold promise in addressing the issues of computational power and energy consumption in artificial intelligence, paving the way for practical applications in the infrared spectrum.