Multi-hop Deflection Routing Algorithm Based on Q-Learning for Energy-Harvesting Nanonetworks

Nanonetworks composed by communicating nano-devices enable new applications in the consumer, biomedical, and environmental fields. Three main characteristics introduce strict requirements for routing protocols design for nanonetworks, namely, short transmission range at Terahertz (THz) frequency (0.1-10 THz), fluctuations in the energy of nano-nodes due to the energy harvesting processes and very limited memory/buffer size of nano-nodes. In this paper, a multi-hop deflection routing algorithm based on Q-learning for energy-harvesting nanonetworks (MDRQEN) is proposed to guarantee the network energy efficiency, while ensuring a low packet loss probability. First, a deflection table is introduced to deflect the packets when the next hop nano-nodes are unavailable due to energy or memory/buffer constraints. Then, a Q-learning scheme is proposed to update the routing table and deflection table by utilizing the reward information contained in the forwarded packet from the previous nano-node. In the Q-learning update scheme, packet deflection ratio, packet loss ratio, packet hop count and node energy status of nano-nodes are taken into consideration. As numerically shown through extensive simulations in Network Simulator 3 (NS-3), the proposed MDRQEN algorithm can achieve a better packet delivery ratio and energy efficiency than random routing algorithm, flooding routing algorithm and the MDRQEN algorithm without the Q-learning update scheme.