Communication Efficient Distributed Estimation Over Directed Random Graphs

Recently, a communication efficient recursive distributed estimator, $C\mathcal{R}\mathcal{E}\mathcal{D}\mathcal{O}$, has been proposed, that utilizes increasingly sparse randomized bidirectional communications. $\lt p\gt C\mathcal{R}\mathcal{E}\mathcal{D}\mathcal{O}$ achieves order-optimal $O(1/t)$ mean square error (MSE) rate in the number of per-node processed samples t, and a $\lt p\gt O(1/C_{t}^{2-\zeta})$ MSE rate in the number of per-node communications, where $\zeta \gt 0$ is arbitrarily small. In this paper, we present directed $C\mathcal{R}\mathcal{E}\mathcal{D}\mathcal{O}, \mathcal{D}-C\mathcal{R}\mathcal{E}\mathcal{D}\mathcal{O}$ for short-a distributed recursive estimator that utilizes directed increasingly sparse communications. We show that $\mathcal{D}-C\mathcal{R}\mathcal{E}\mathcal{D}\mathcal{O}$ further dramatically improves communication efficiency, achieving the $O(1/c\mathcal{T})$ communication MSE rate with arbitrarily high exponent $\kappa$, while keeping the order-optimal $O(1/t)$ sample-wise MSE rate. Numerical examples on real data sets confirm our results.