Adaptive Switching for Multimodal Underwater Acoustic Communications Based on Reinforcement Learning

The underwater acoustic (UWA) channel is a complex and stochastic process with large spatial and temporal dynamics. This work studies the adaptation of the communication strategy to the channel dynamics. Specifically, a set of communication strategies are considered, including frequency shift keying (FSK), single-carrier communication, and multicarrier communication. Based on the channel condition, a reinforcement learning (RL) algorithm, the Depth Determined Strategy Gradient (DDPG) method along with a Gumbel-softmax scheme is employed for intelligent and adaptive switching among those communication strategies. The adaptive switching is performed on a transmission block-by-block basis, with the goal of maximizing a long-term system performance. The reward function is defined based on the energy efficiency and the spectral efficiency of the communication strategies. Simulation results reveal that the proposed method outperforms a random selection method in time-varying channels.