Sensitivity Analysis of Neural Network Hyperparameters for Chromatic Dispersion Compensation in Optical Transmissions

Abstract

To verify and validate the use of machine learning techniques for chromatic dispersion compensation, we developed an end-to-end recurrent neural network (RNN) to replace the digital signal processing (DSP) blocks used in optical transmission and reception. We also evaluated the sensitivity of the developed networks to certain hyperparameters. Our analysis indicated that the number of neurons and the number of epochs were the most impactful parameters, and we also observed that using lower values for these parameters resulted in performance that was closer to that of a conventional DSP implementation.