Fiber channel modeling based on CGAN and three-dimensional geometric shaping based on E2EDL

Abstract

To optimize the complex nonlinear effects in optical communication systems, this paper introduces channel modeling and three-dimensional (3D) geometric shaping based on end-to-end deep learning (E2EDL), which includes the autoencoder and conditional generative adversarial network (CGAN). The CGAN is employed for fiber channel modeling, and the autoencoder realizes 3D geometric shaping of carrierless amplitude phase (CAP)-16, 32, and 64, which achieves an automatic and smart nonlinear control of optical communication systems. However, compared with two-dimensional (2D) modulation, the 3D modulation requires a larger training dataset, thereby increasing the complexity of the model. By pre-training the transceiver of the autoencoder, the training process becomes more stable, the training time is reduced, and the bit error rate (BER) of the system is further optimized. The results indicate that the running time of the pre-trained system is 55 % less than the traditional system. Compared with 2D modulation, the BER performance of 3D CAP-16/32/64 is improved by an order of magnitude. It demonstrates the effectiveness of the proposed deep learning framework in mitigating nonlinear impairments and achieving global performance optimization in fiber-optic communication systems.