Distributed empirical risk minimization over directed graphs

Abstract

In this paper, we present stochastic optimization for empirical risk minimization over directed graphs. Using a novel information fusion approach that utilizes both row- and column-stochastic weights simultaneously, we propose $\mathcal{S}\mathcal{A}\mathcal{B}$, a decentralized stochastic gradient method with gradient tracking, and show that the proposed algorithm converges linearly to an error ball around the optimal solution with a constant step-size. We provide a sketch of the convergence analysis as well as the generalization of the proposed algorithm. Finally, we illustrate the theoretical results with the help of experiments with real data.