Underwater Acoustic Adaptive Modulation with Reinforcement Learning and Channel Prediction

Abstract

We present our ongoing work on reinforcement learning based adaptive modulation for underwater acoustic (UWA) communication. As the long propagation delay, the feedback channel state information (CSI) will be outdated for UWA adaptive modulation. Reinforcement learning can learn the optimal transmission mode even with the outdated CSI. Furthermore, this paper employs long short-term memory neural network to predict CSI for updating learning table, and then selects modulation mode by ϵ– greedy algorithm. The simulation results revealed that the network throughput of proposed method is improved under error bit rate constraint, which is compared to Q-learning with state transition probability predicted CSI and direct feedback CSI in time varying UWA channel.