QRCF: A new Q-learning-based routing approach using a smart cylindrical filtering system in flying ad hoc networks

To ensure reliable data transmission in flying ad hoc networks (FANETs), efficient routing protocols are necessary to establish communication paths in FANETs. Recently, reinforcement learning (RL), particularly Q-learning, has become a promising approach for overcoming challenges faced by traditional routing protocols due to its capacity for autonomous adaptation and self-learning. This study presents a Q-learning-based routing strategy, enhanced by an innovative cylindrical filtering technique, named QRCF in FANETs. In QRCF, the dissemination interval of hello packets is adaptively adjusted based on the connection status of nearby UAVs. Then, this routing process leverages Q-learning to discover reliable and stable routes, using a state set refined by the cylindrical filtering technique to accelerate the search for the optimal path in the network. Afterward, the reward value is computed using metrics such as relative speed, connection time, residual energy, and movement path. Finally, QRCF is deployed in the network simulator 2 (NS2), and its performance is evaluated against three routing schemes, QRF, QFAN, and QTAR. These evaluations are presented based on the number of UAVs and their speed. In general, when changing the number of nodes, QRCF improves energy usage (about 5.01%), data delivery ratio (approximately 1.20%), delay (17.71%), and network longevity (about 3.21%). However, it has a higher overhead (approximately 10.91%) than QRF. Moreover, when changing the speed of UAVs in the network, QRCF improves energy usage (about 4.94%), data delivery ratio (approximately 2.36%), delay (about 17.5%), and network lifetime (approximately 8.75%). However, it increases routing overhead (approximately 15.47%) in comparison with QRF.