Quasi-Newton updating for large-scale distributed learning

IF 3.1 1区数学 Q1 STATISTICS & PROBABILITY

Journal of the Royal Statistical Society Series B-Statistical Methodology Pub Date : 2023-06-10 DOI:10.1093/jrsssb/qkad059

Shuyuan Wu, Danyang Huang, Hansheng Wang

引用次数: 0

Abstract

Abstract Distributed computing is critically important for modern statistical analysis. Herein, we develop a distributed quasi-Newton (DQN) framework with excellent statistical, computation, and communication efficiency. In the DQN method, no Hessian matrix inversion or communication is needed. This considerably reduces the computation and communication complexity of the proposed method. Notably, related existing methods only analyse numerical convergence and require a diverging number of iterations to converge. However, we investigate the statistical properties of the DQN method and theoretically demonstrate that the resulting estimator is statistically efficient over a small number of iterations under mild conditions. Extensive numerical analyses demonstrate the finite sample performance.

查看原文本刊更多论文

大规模分布式学习的准牛顿更新

分布式计算对现代统计分析至关重要。在此，我们开发了一个具有出色统计，计算和通信效率的分布式准牛顿(DQN)框架。在DQN方法中，不需要Hessian矩阵反演和通信。这大大降低了该方法的计算和通信复杂度。值得注意的是，现有的相关方法只分析数值收敛性，并且需要发散迭代数才能收敛。然而，我们研究了DQN方法的统计性质，并从理论上证明了在温和的条件下，在少量迭代中得到的估计器是统计有效的。大量的数值分析证明了有限样本的性能。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of the Royal Statistical Society Series B-Statistical Methodology 数学-统计学与概率论

CiteScore

8.80

自引率

0.00%

发文量

审稿时长

>12 weeks

期刊介绍： Series B (Statistical Methodology) aims to publish high quality papers on the methodological aspects of statistics and data science more broadly. The objective of papers should be to contribute to the understanding of statistical methodology and/or to develop and improve statistical methods; any mathematical theory should be directed towards these aims. The kinds of contribution considered include descriptions of new methods of collecting or analysing data, with the underlying theory, an indication of the scope of application and preferably a real example. Also considered are comparisons, critical evaluations and new applications of existing methods, contributions to probability theory which have a clear practical bearing (including the formulation and analysis of stochastic models), statistical computation or simulation where original methodology is involved and original contributions to the foundations of statistical science. Reviews of methodological techniques are also considered. A paper, even if correct and well presented, is likely to be rejected if it only presents straightforward special cases of previously published work, if it is of mathematical interest only, if it is too long in relation to the importance of the new material that it contains or if it is dominated by computations or simulations of a routine nature.