Energy-Latency Tradeoffs for Service Placement Based on Reinforcement Learning in Edge Computing

Abstract

Microservice technology, as a flexible application architecture, has gained wide popularity in the field of Internet of Things (IoT). IoT applications are highly sensitive to latency, making it crucial to place microservices on appropriate edge servers in an edge computing environment. Failure to do so can significantly impact service quality and degrade user experience, posing a major challenge. Addressing the aforementioned issues, this paper proposes a multiobjective service deployment strategy for IoT devices based on reinforcement learning. The goal is to minimize service access delay for IoT devices and reduce the average energy consumption of edge servers in the context of mobile edge computing. To achieve this, we first establish a stochastic optimization model using the Markov decision process (MDP) framework to handle service deployment and resource allocation dynamically. This model captures key characteristics such as heterogeneity in edge server capabilities, dynamic geographic information of IoT devices, and uncertainty in microservice requests. To overcome challenges related to dimensionality, slow convergence, and the exploration–exploitation tradeoff in traditional reinforcement learning algorithms, we introduce deep reinforcement learning into the optimization of microservice deployment. Specifically, we propose the use of deep deterministic policy gradient (DDPG) to obtain a near-optimal service deployment strategy without manual instructions. DDPG leverages the depth of the network to guide policy gradients and generate solutions that effectively balance exploration and exploitation. To evaluate the proposed approach, we implement the DPG-MSP (DDPG-based MicroService Placement) algorithm using real datasets and synthetic data. Comparative analysis with existing microservice deployment algorithms demonstrates the superiority of DDPG-MSP in terms of performance, robustness, and scalability.