A Deep Q-Learning Design for Energy Harvesting QoS Routing in IoT-enabled Cognitive MANETs

In this paper, we propose an energy harvesting quality-of-service (EH-QoS) routing protocol based on a deep Q-learning design in Internet-of-Things-enabled cognitive radio mobile ad hoc networks (IoT-CMANETs), where mobile nodes harvest energy from a multiple antennas power beacon for their routing and data transmission processes. A deep Q-learning network (DQN) is proposed to establish a QoS route, which avoids the affected region of a primary user. In the forwarding route request (RREQ) process, relying on the designed DQN, the proposed EH-QoS routing protocol unicasts a RREQ packet to the neighbor associated with a minimum $Q^{\ast} -$ value satisfying energy, queue size of each node, the number of hops, and cognitive radio constraints. The $Q^{\ast} -$ value of each link is obtained by optimizing joint residual energy and speed of all nodes belonging to this link. Simulation results show that the proposed EH-QoS routing protocol outperforms the state-of-the-art routing protocols in terms of control overhead, packet delivery ratio, routing delay, and energy consumption, arising as an effective protocol in IoT-CMANETs.