Clustering-Based Reinforcement Learning Routing Protocol with Low Complexity for Underwater Acoustic Sensor Networks

With the increase in the number of relay nodes in large-scale underwater acoustic sensor networks (UWA-SNs), the learning costs of routing protocols based on reinforcement learning algorithms continue to increase. In this paper, we take the Q-learning (QL) algorithm as the example of the reinforcement learning algorithm, combined with the clustering algorithm based on genetic simulated annealing algorithm (SAGAFCM), and propose an improved QL routing protocol (IQLR) aimed at reducing the time complexity of QL-based routing algorithm. The core idea is to preprocess the underwater relay nodes before applying the traditional QL algorithm for routing optimization. The simulation results show that compared with the un-preprocessed/traditional QL routing algorithm (UQLR), the proposed IQLR can greatly reduce the time cost of the QL-based routing algorithm while ensuring that the transmission quality does not change much.