A multi-agent enhanced DDPG method for federated learning resource allocation in IoT

Abstract

In the Internet of Things (IoT), federated learning (FL) is a distributed machine learning method that significantly improves model performance by utilizing local device data for collaborative training. However, applying FL in IoT also presents new challenges: the significant differences in computing and communication capabilities among IoT devices and the limited resources make efficient resource allocation crucial. This paper proposes a multi-agent enhanced deep deterministic policy gradient method (MAEDDPG) based on deep reinforcement learning to obtain the optimal resource allocation strategy. Firstly, MAEDDPG introduces long short-term memory networks to address the local observation problem in multi-agent settings. Secondly, noise networks are employed during training to enhance exploration, preventing the model from getting stuck in local optima. Finally, an enhanced double critic network is designed to reduce the error in value function estimation. MAEDDPG effectively obtains the optimal resource allocation strategy, coordinating the computing and communication resources of various IoT devices, thereby balancing FL training time and IoT device energy consumption. The experimental results show that the proposed MAEDDPG method outperforms the state-of-the-art method in IoT, reducing the average system cost by 12.4%.