QUERA: Q-Learning RPL Routing Mechanism to Establish Energy Efficient and Reliable Communications in Mobile IoT Networks

Abstract

Resource-limited mobile IoT networks are a dynamic, and uncertain wireless communicating system. In such systems, the standard RPL routing protocol cannot select long-lasting communication links due to not employing mobility-aware metrics, e.g., direction and speed of movements. While several classical heuristic approaches exist to improve PDR in RPL-based mobile networks, their solutions cannot adapt to alterations of the mobile topology. Hence, in this paper, by mapping the routing problem in mobile and resource-limited networks into an infinite-time horizon MDP, an energy-aware and reliable RPL-based routing mechanism based on Q-learning is proposed to improve PDR in mobile IoT networks. This routing mechanism, which is called QUERA, utilizes mobility and quality-aware metrics, including Time-to-Reside (TTR), ETX, and RSSI. Furthermore, QUERA probes and maintains stable candidates based on its neighbor table management policy. These two aspects mitigate the need for retransmissions due to packet loss leading to less energy dissipation. According to evaluations, QUERA improves energy consumption by up to 50% against the state-of-the-art. The efficiency of QUERA is also evaluated in terms of power distribution diagram, which shows significant improvement in the lifetime of IoT devices. It has also been observed that QUERA improves PDR in mobile networks by up to 12%.