Extending battery lifespan in IoT extreme sensor networks through collaborative reinforcement learning-powered task offloading

Abstract

The use of wireless sensor networks (WSN) is increasingly widespread in the Internet of Things domain. Additionally, since the onset of the edge computing paradigm that brings the cloud closer to devices, these networks have seen improvements in battery lifetime and processing time, particularly in extreme edge architectures where network resources are more limited. Meanwhile, AI and machine learning techniques have been expanding across various domains to optimize different decision-making processes, including the task assignment problem in computation offloading. This article employs reinforcement learning (RL) techniques to address the task offloading problem, aiming to extend the lifespan of a WSN. To achieve this, a distributed multi-agent Q-learning algorithm is proposed, where sensor nodes (SNs) make collaborative decisions towards a common goal, avoiding selfish decision-making. The proposed algorithm is compared with two other state-of-the-art solutions, that is, a well-known Q-learning algorithm that allows centralized estimation of the Q-table before distributing it to the network’s sensor nodes (SNs), and a similar implementation of this algorithm but using Deep Q-learning, which theoretically should achieve the best results. The outcomes show that the multi-agent RL algorithm improves performance when it takes other nodes in the network into account in its decisions, being the most suitable solution to be embedded in resource-constrained devices. Although it still achieves worse results than the Deep Q-learning algorithm, the latter involves much greater difficulties for implementation in real devices.