Privacy-Preserving Push-Pull Method for Decentralized Optimization via State Decomposition

Abstract

Distributed optimization is manifesting great potential in multiple fields, e.g., machine learning, control, resource allocation, etc. Existing decentralized optimization algorithms require sharing explicit state information among the agents, which raises the risk of private information leakage. To ensure privacy security, combining information security mechanisms, such as differential privacy and homomorphic encryption, with traditional decentralized optimization algorithms is a commonly used means. However, this may either sacrifice optimization accuracy or incur a heavy computational burden. To overcome these shortcomings, we develop a novel privacy-preserving decentralized optimization algorithm, named PPSD, that combines gradient tracking with a state decomposition mechanism. Specifically, each agent decomposes its state associated with the gradient into two substates. One substate is used for interaction with neighboring agents, and the other substate containing private information acts only on the first substate and thus is entirely agnostic to other agents. When the objective function is smooth and satisfies the Polyak-Łojasiewicz (PL) condition, PPSD attains an $R$ -linear convergence rate. Moreover, the algorithm can preserve the normal agents' private information from being leaked to honest-but-curious attackers. Simulations further confirm the results.