Energy-efficient Nonuniform Cluster-based Routing Protocol with Q-Learning for UASNs

Abstract

Underwater Acoustic Sensor Networks (UASNs) are widely used in various fields. The limited energy of underwater nodes and the difficulty of replenishment constrain the lifetime of UASNs. Thus, it is important to effectively balance energy consumption and design a routing protocol that ensures efficient data transmission and extends network lifetime. Cluster routing protocol is widely recognized as an energy efficient strategy for UASNs. However, it faces challenges including “hotspot” issues caused by nodes frequently acting as cluster heads (CHs) and forwarding packets, as well as energy inefficiency resulting from packet conflicts and redundant transmissions. Therefore, we propose an Energy-efficient Nonuniform Cluster-based Routing Protocol with Q-Learning (ENCRQ) to balance energy consumption and improve packet forwarding efficiency. In the CH election phase, a “CH competitiveness” function is created based on node weighted density and residual energy. Nodes outside the sink neighborhood adaptively compete for CHs based on this function, aiming to achieve an uneven distribution of CHs and balance energy consumption. Meanwhile, nodes within the sink neighborhood remain sleeping to further reduce energy consumption. In the inter-cluster routing phase, the forwarding area is hierarchically divided based on node depth and distance to the sink node. The optimal forwarding area is adaptively adjusted to form the next-hop candidate set. On this basis, holding time is designed for candidate nodes based on Q-Learning technique to prioritize forwarding, minimizing packet conflicts and redundant transmissions, thereby enhancing network transmission efficiency. Simulation results show that compared with LEACH, QELAR and QHUC, ENCRQ has significant improvement in terms of network lifetime, energy balance and energy efficiency.