Optimized convergence of stochastic gradient descent by weighted averaging

IF 1.4 3区数学 Q3 COMPUTER SCIENCE, SOFTWARE ENGINEERING

Optimization Methods & Software Pub Date : 2024-02-05 DOI:10.1080/10556788.2024.2306383

Melinda Hagedorn, Florian Jarre

引用次数: 0

Abstract

Under mild assumptions stochastic gradient methods asymptotically achieve an optimal rate of convergence if the arithmetic mean of all iterates is returned as an approximate optimal solution. Howev...

查看原文本刊更多论文

通过加权平均优化随机梯度下降的收敛性

在温和的假设条件下，如果所有迭代次数的算术平均值作为近似最优解返回，随机梯度方法就会逐渐达到最佳收敛速度。然而...

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Optimization Methods & Software 工程技术-计算机：软件工程

CiteScore

4.50

自引率

0.00%

发文量

审稿时长

7 months

期刊介绍： Optimization Methods and Software publishes refereed papers on the latest developments in the theory and realization of optimization methods, with particular emphasis on the interface between software development and algorithm design. Topics include: Theory, implementation and performance evaluation of algorithms and computer codes for linear, nonlinear, discrete, stochastic optimization and optimal control. This includes in particular conic, semi-definite, mixed integer, network, non-smooth, multi-objective and global optimization by deterministic or nondeterministic algorithms. Algorithms and software for complementarity, variational inequalities and equilibrium problems, and also for solving inverse problems, systems of nonlinear equations and the numerical study of parameter dependent operators. Various aspects of efficient and user-friendly implementations: e.g. automatic differentiation, massively parallel optimization, distributed computing, on-line algorithms, error sensitivity and validity analysis, problem scaling, stopping criteria and symbolic numeric interfaces. Theoretical studies with clear potential for applications and successful applications of specially adapted optimization methods and software to fields like engineering, machine learning, data mining, economics, finance, biology, or medicine. These submissions should not consist solely of the straightforward use of standard optimization techniques.