An Edge Federated MARL Approach for Timeliness Maintenance in MEC Collaboration

Mobile edge computing (MEC) has been widely studied to provide new schemes for communication-computing systems such as industrial Internet of Things (IoTs), vehicular networks, smart city applications, etc. In this work, we mainly investigate on the timeliness maintenance of the MEC systems where the freshness of the data and computation tasks plays a significant role. We firstly formulate the average age of information (AoI) minimization problem of the UAV-assisted MEC systems. To maintain the system timeliness, we propose a novel multi-agent reinforcement learning (MARL) approach, called edge federated multi-agent actor-critic (MAAC), for joint trajectory planning, data scheduling and resource management in the investigated MEC systems. Through the proposed online learning method, edge devices and center controller learn the interactive policies through local observations and carry out the model-wise communication. We build up a simulation platform for time sensitive MEC systems as a gym environment module and implement the proposed algorithm. Furthermore, the comparisons with a popular MARL solution, MADDPG, show that the proposed approach achieves better performance in terms of data freshness and system stability.