Differential Privacy Enabled Robust Asynchronous Federated Multitask Learning: A Multigradient Descent Approach

The federated learning (FL) technique can provide a promising solution for the timely training of a deep learning model with the critical requirement of privacy protection. However, the existing FL frameworks still confront challenging issues including heterogeneous data sources, edge device heterogeneity, sensitive information leakage, nonconvex loss, and communication resource constraints which place obstacles in terms of practicality. In this article, first, a federated multitask learning (FedMTL) approach is introduced to reformulate the FL model as a multiobjective optimization problem which results in federated multigradient descent algorithm (FedMGDA) with a better model personalization against data heterogeneity and Byzantine attack. Second, a new semi-asynchronous model aggregation method is developed to asynchronously aggregate small partial clients for compensating impacts of the straggler and staleness. Third, a distributed differential privacy technique is applied to enhance the privacy protection of asynchronous FedMGDA with the convergence guarantee where the convergence analysis of differentially private asynchronous federated multiple gradient descent algorithm (DP-AsynFedMGDA) is studied for both the convex and the nonconvex loss functions. Empirical examples and comparative studies are presented to illustrate the effectiveness of the proposed DP-AsynFedMGDA.