Private and communication-efficient edge learning: a sparse differential gaussian-masking distributed SGD approach

Abstract

With the rise of machine learning (ML) and the proliferation of smart mobile devices, recent years have witnessed a surge of interest in performing ML in wireless edge networks. In this paper, we consider the problem of jointly improving data privacy and communication efficiency of distributed edge learning, both of which are critical performance metrics in wireless edge network computing. Toward this end, we propose a new distributed stochastic gradient method with sparse differential Gaussian-masked stochastic gradients (SDM-DSGD) for non-convex distributed edge learning. Our main contributions are three-fold: i) We theoretically establish the privacy and communication efficiency performance guarantee for our SDM-DSGD method, which outperforms all existing works; ii) We propose a generalized differential-coded DSGD update, which enables a much lower transmit probability for gradient sparsification, and provides an [EQUATION] convergence rate; and iii) We reveal theoretical insights and offer practical design guidelines for the interactions between privacy preservation and communication efficiency - two conflicting performance goals. We conduct extensive experiments with a variety of learning models on MNIST and CIFAR-10 datasets to verify our theoretical findings.