Distributed Machine Learning over Directed Network with Fixed Communication Delays

Abstract

In this paper, we present a distributed machine learning algorithm over a network with fixed-delay tolerance. The network is directed and strongly connected. The training dataset is distributed to all agents in the network. We combine the distributed convex optimization (which utilizes double linear iterations) and corresponding machine learning algorithm. Each agent can only access its own local dataset. Suppose the delay between any pair of agents is time-invariant. The simulation shows that our algorithm is able to work under delayed transmission, in the sense that over time at each agent t the ratio of the estimate value xi(t) and scaling variable yi(t) can converge to the optimal point of the global cost function corresponding to the machine learning problem.