A Q-Learning and Data Priority-Based Routing Protocol with Dynamic Computing Cluster Head for Underwater Acoustic Sensor Networks

Abstract

Underwater acoustic sensor network (UASN) is a promising underwater networking technology for wide applications, but there is an urgent need to design reliable and low power consumption routing protocols for UASN to extend network lifetime due to the limited energy supply. In this paper, we propose a Q-learning and data priority-based routing protocol with dynamic computing cluster head (QD-DCR) to extend the network lifetime of UASN. In QD-DCR protocol, the underwater nodes are clustered and set the cluster head (CH) nodes, which are only responsible for computing the optimal path of data transmission and the storage of Q-value table, while the non-CH nodes are responsible for data transmission. Meanwhile, according to the data priority, we design different data transmission methods that can effectively use the limited resources of UASN to transmit urgent data. To further make the residual energy of sensor nodes evenly distributed, we also design the dynamic selection of CH node, which can avoid the potential energy holes. In addition, we adopt Q-learning to determine the optimal next hop instead of the greedy next hop in a cluster. We also define an action utility function that takes into account both residual energy and node depth to extend the network lifetime by distributing the residual energy evenly. Simulation results show that the proposed QD-DCR protocol can effectively extend the network lifetime compared with a classic lifetime-extended routing protocol (QELAR), while alleviating the issue of uneven distribution of the residual energy in the network.