Design of Incentive Mechanism for Node Collaboration in Hierarchical Federated Learning Based on Deep Reinforcement Learning

Abstract

With the rapid development of artificial intelligence, big data, and distributed computing technologies, hierarchical federated learning has emerged as a widely studied distributed machine learning framework. In hierarchical federated learning, edge servers are deployed between cloud servers and mobile devices, efficiently receiving local models from nearby mobile devices and performing edge model aggregation. Node collaboration in hierarchical federated learning can reduce training costs and improve model quality while protecting data privacy. However, data security risks and resource consumption during model training can reduce the willingness of mobile devices to participate. Additionally, collaborative nodes are often heterogeneous, facing issues such as skewed datasets and imbalanced capabilities. Therefore, this paper proposes a deep reinforcement learning-based incentive mechanism for node collaboration, aimed at maximizing node benefits. A node collaboration strategy optimization model is then constructed using the Markov decision process framework, and the NCIA algorithm, based on deep reinforcement learning networks, is designed. Finally, through extensive simulation experiments, the proposed NCIA algorithm is demonstrated to improve model accuracy by 5.28% and 14.22% compared with the CCEG and FedAvg algorithms, respectively.