Reinforcement learning-based load balancing for heavy traffic Internet of Things

Abstract

Aiming to large-scale data transmission requirements of resource-constrained IoT (Internet of Things) devices, the routing protocol for low power lossy network (RPL) is expected to handle the load imbalance and high energy consumption in heavy traffic scenarios. This paper proposes a novel RPL routing optimization Algorithm based on deep Reinforcement Learning (referred to as RARL), which employs the centralized training and decentralized execution architecture. Hence, the RARL can provide the intelligent parent selection policy for all nodes while improving the training efficiency of deep reinforcement learning (DRL) model. Furthermore, we integrate a new local observation into the RARL by exploiting multiple routing metrics and design a comprehensive reward function for enhancing the load-balance and energy efficiency. Meanwhile, we also optimize the Trickle timer mechanism for adaptively controlling the delivery of DIO messages, which further improves the interaction efficiency with environment of DRL model. Extensive simulation experiments are conducted to evaluate the effectiveness of RARL under various scenarios. Compared with some existing methods, the simulation results demonstrate the significant performance of RARL in terms of network lifetime, queue loss ratio, and packet reception ratio.