Adaptive weighted approach for high-dimensional statistical learning and inference

Abstract

We propose a new weighted average estimator for high-dimensional parameters under the distributed learning system, where the weight assigned to each coordinate across different agents is precisely proportional to the inverse of the variance of the local estimates for that agent. This strategy, on the one hand, enables the new estimator to achieve a minimal mean squared error, consistent with the current state-of-the-art one-shot distributed learning method, and on the other hand, it maintains remarkably low communication costs, as each agent is required to transmit only two vectors to the central server. We further demonstrate the effectiveness of the new estimator by investigating the error bound and the asymptotic properties of the estimation, as well as the numerical performance on simulated examples and a real-world data application.