Double DQN-based Efficient Quality of Service Routing protocol in Internet of Underwater Things with mobile nodes

Abstract

Internet of Underwater Things (IoUT) refers to a self-organizing network comprised of energy limited sensor nodes to collect underwater sensory information, which has become a popular research topic due to its both military and commercial applications. How to transmit sensory data to sink nodes with wireless acoustic communication is a great challenge to Internet of Underwater Things. This paper proposes a Double deep $Q$ Network-based Efficient Quality of Service Routing protocol (DQN-EQSR in short) as the routing strategy of IoUT, which can improve transmission performance of wireless data delivery in IoUT. First of all, this paper establishes a wireless routing system for IoUT, where the applied Quality of Service (QoS) metric consists of packet delivery ratio, network life cycle, and end-to-end delay. Then, this paper establishes multi-dimensional state, multi-dimensional action, and multi-factor reward function for each sensor node. The multi-dimensional state includes node information and packet information, multi-dimensional action includes relay nodes selection and acoustic communication mode, and multi-factor reward function includes many factors such as energy cost, link quality cost, delay cost and packet priority. Moreover, a double Deep $Q$ Network (DQN) is provided to evaluate the action value of nodes, where DQN1 is used to determine relay nodes, and DQN2 is used to determine acoustic communication controls. In addition, this paper conducts many simulations to prove the effectiveness of DQN-EQSR algorithm. Extensive simulation results show that the proposed DQN-EQSR algorithm outperforms other protocols in terms of packet forwarding hops, alive nodes ratio, residual energy ratio, average end-to-end delay and packet delivery ratio.