IoTSim-Osmosis-MARL: Towards Multi-Agent Reinforcement Learning Osmotic Computing

Abstract

Internet of Things systems exist in various areas of our everyday life. Data from sensors installed in smart cities and homes is processed in edge and cloud computing centers, providing several benefits that improve our lives. The group of devices might cooperate to fulfill desired goals, trying to preserve their resources and handle the failures of the devices. The paper presents the multi-agent reinforcement learning (MARL) extension to the osmotic computing simulation framework, enabling direct interactions between IoT devices. The proposed approach allows IoT devices to operate autonomously and cooperatively in dynamic environments, reducing the need for manual intervention and enabling resilient, energy-efficient sensing coverage. We discuss the multi-agent sensing coverage problem as one directly applicable to the IoT sensing systems. We identify the challenges posed to the framework and analyze management algorithms for cooperating osmotic agents. In the evaluation, we demonstrate that cooperation between devices enables the self-autonomous behavior of IoT systems. A case study yields promising results, showing that the adaptation of sensors may allow them to replace each other by lowering energy usage (up to 25% reduction), increasing global coverage (from 74% to 92%), and preserving battery life by dynamically adjusting their sensing range. Finally, the presented framework is a novel contribution that combines MARL environments with IoT systems simulation, enabling future research in this field.