Adaptive modulation switching strategy based on Q-learning for underwater acoustic communication channel

Abstract

Adaptive modulation techniques which use different modulation methods to match time-varying channel-state information (CSI) are widely used to improve the robustness of underwater acoustic communication links. We propose a novel Q-learning-based adaptive modulation-switching strategy. During the communication, the CSI is estimated from the handshake signals or training sequences, etc., in each communication block at the receiver, and the CSI is then sent back to the transmitter at the feedback channel. The transmitter adaptively selects an appropriate modulation method via Q-learning. The utility of the proposed strategy is based on essential communication parameters such as signal-to-noise ratio (SNR), energy consumption and bit error rate (BER). We conducted simulation tests to testify to the performance of the proposed strategy. Two modulation methods, (i.e., multi-carrier multiple-frequency shift keying (MC-MFSK) and quadrature phase shift keying (QPSK)), which have distinct features in underwater acoustic channel (UWA) communication, are considered in the tests. The performance is evaluated with simulations for the shallow-sea tide condition, and the results show that our proposed adaptive modulation-switching strategy performs well compared with a random selection strategy.