Iteration and stochastic first-order oracle complexities of stochastic gradient descent using constant and decaying learning rates

IF 1.6 3区数学 Q2 MATHEMATICS, APPLIED

Optimization Pub Date : 2024-06-19 DOI:10.1080/02331934.2024.2367635

Kento Imaizumi, Hideaki Iiduka

引用次数: 0

Abstract

The performance of stochastic gradient descent (SGD), which is the simplest first-order optimizer for training deep neural networks, depends on not only the learning rate but also the batch size. T...

查看原文本刊更多论文

使用恒定和衰减学习率的随机梯度下降的迭代和随机一阶甲骨文复杂性

随机梯度下降（SGD）是用于训练深度神经网络的最简单的一阶优化器，其性能不仅取决于学习率，还取决于批量大小。T...

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Optimization 数学-应用数学

CiteScore

4.50

自引率

9.10%

发文量

146

审稿时长

4.5 months

期刊介绍： Optimization publishes refereed, theoretical and applied papers on the latest developments in fields such as linear, nonlinear, stochastic, parametric, discrete and dynamic programming, control theory and game theory. A special section is devoted to review papers on theory and methods in interesting areas of mathematical programming and optimization techniques. The journal also publishes conference proceedings, book reviews and announcements. All published research articles in this journal have undergone rigorous peer review, based on initial editor screening and anonymous refereeing by independent expert referees.