Dual-Agent DRL-Based Service Placement, Task Scheduling, and Resource Allocation for Multi-Sensor and Multi-User Edge Computing Networks

Abstract

Multi-sensor and multi-user edge computing networks can support various data-intensive Internet of Things (IoT) applications, which exhibit the characteristic of a task-data-decoupled pattern. In this scenario, tasks generated by users can be scheduled to edge servers (ESs), where the ESs compute the task results and return them to the users. Meanwhile, a large number of sensors collect and upload data to the ESs to meet the requirements of task processing. However, existing works mainly consider the task-data-coupled pattern, overlooking the cost associated with sensor data collection processes. Therefore, we propose a joint optimization problem involving service placement, task scheduling, and resource allocation to minimize the total system cost, defined as the weighted sum of delay and energy consumption of each user and sensor. We jointly optimize service placement, user task scheduling, transmit power allocation for sensors and users, computing resource allocation for both the ESs and the cloud server (CS), and transmission rate allocation for ES-ES and ES-CS connections. Considering the differences in the update frequencies of the optimization variables, we propose a dual-agent Deep Reinforcement Learning (DRL) algorithm, which utilizes two SD3 (Softmax Deep Double Deterministic Policy Gradients)-based DRL agents to make service placement and task scheduling decisions asynchronously, while embedding two optimization subroutines to solve the optimal transmit power allocation, computing resource and transmission rate allocations using numerical methods. The complexity and convergence of the algorithm are analyzed and extensive experiments are conducted in 8 different scenarios, demonstrating the superiority of our scheme compared to three other reference schemes.