Optimization of digital back-propagation for coherent optical fiber communication systems using fourth-order Runge-Kutta in the interaction picture method.

The digital back-propagation (DBP) is an algorithm that can equalize the chromatic dispersion and nonlinearity in the coherent optical fiber communication system. However, the nonlinear equalization effect of traditional split-step Fourier method (SSFM)-based DBP is limited. This paper replaces the SSFM in DBP algorithm with the fourth-order Runge-Kutta in the interaction picture (RK4IP) method, and employs the Bayesian optimization algorithm (BOA) to optimize the coefficients in RK4IP-based DBP algorithm, then compares it with SSFM-based DBP algorithm, which is also optimized using BOA. The experimental results of 9×100 km standard single-mode fiber (SSMF) 20 Gbaud 16QAM transmission system demonstrate that the coefficient-optimized RK4IP (CO-RK4IP)-based DBP algorithm can achieve the maximum improvement of 0.89 dB in the Q-factor compared to the coefficient-optimized SSFM (CO-SSFM)-based DBP algorithm at equivalent complexity, proving that CO-RK4IP is an effective recursive method for DBP algorithm.