CNN based intra channel nonlinear equalization method for WDM CO-OFDM systems

Polarization-division multiplexing wavelength-division multiplexing coherent optical orthogonal frequency-division multiplexing (PDM-WDM CO-OFDM) systems have emerged as a key technology in optical communications due to their high spectral efficiency and ability to resist frequency selective fading. However, in long-distance or high-data-rate transmission scenarios, signal distortion caused by fiber nonlinear effects significantly degrades system performance. Traditional nonlinear compensation methods, such as the high-order Volterra series, can address some of these challenges but are limited by high computational complexity and insufficient accuracy, making them unsuitable for real-time transmission requirements. To address these challenges, we propose a convolutional neural network (CNN)-based nonlinear equalization method. By introducing a sliding window mechanism, the global signal is divided into local segments, enabling the model to efficiently capture local nonlinear features while maintaining low computational complexity. Additionally, CNNs leverage parameter sharing and parallel computation to significantly improve training and inference speeds, overcoming the computational inefficiencies and sequential nature of recurrent neural networks (e.g., Bi-LSTM) in handling time-series data. Simulation results demonstrate that, for a 16-QAM modulated PDM-WDM CO-OFDM system with a data rate of 850 Gb/s, the proposed method improves the maximum transmission distance by 100 % compared to the Volterra series model. Furthermore, while achieving comparable bit error rate (BER) performance to the Bi-LSTM model, the computational complexity of the CNN-based method is significantly decreased. The proposed CNN method exhibits superior robustness and performance in nonlinear equalization.