用于随机优化的带自适应步长的随机方差降低梯度法

Jing Li, Dan Xue, Lei Liu, Rulei Qi
{"title":"用于随机优化的带自适应步长的随机方差降低梯度法","authors":"Jing Li, Dan Xue, Lei Liu, Rulei Qi","doi":"10.1002/oca.3109","DOIUrl":null,"url":null,"abstract":"In this paper, we propose a stochastic variance reduction gradient method with adaptive step size, referred to as the SVRG-New BB method, to solve the convex stochastic optimization problem. The method could be roughly viewed as a hybrid of the SVRG algorithm and a new BB step mechanism. Under the condition that the objective function is strongly convex, we provide the linear convergence proof of this algorithm. Numerical experiment results show that the performance of the SVRG-New BB algorithm can surpass other existing algorithms if parameters in the algorithm are properly chosen.","PeriodicalId":501055,"journal":{"name":"Optimal Control Applications and Methods","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A stochastic variance reduced gradient method with adaptive step for stochastic optimization\",\"authors\":\"Jing Li, Dan Xue, Lei Liu, Rulei Qi\",\"doi\":\"10.1002/oca.3109\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In this paper, we propose a stochastic variance reduction gradient method with adaptive step size, referred to as the SVRG-New BB method, to solve the convex stochastic optimization problem. The method could be roughly viewed as a hybrid of the SVRG algorithm and a new BB step mechanism. Under the condition that the objective function is strongly convex, we provide the linear convergence proof of this algorithm. Numerical experiment results show that the performance of the SVRG-New BB algorithm can surpass other existing algorithms if parameters in the algorithm are properly chosen.\",\"PeriodicalId\":501055,\"journal\":{\"name\":\"Optimal Control Applications and Methods\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-02-08\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Optimal Control Applications and Methods\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1002/oca.3109\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Optimal Control Applications and Methods","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1002/oca.3109","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

本文提出了一种具有自适应步长的随机方差降低梯度方法,称为 SVRG-New BB 方法,用于解决凸随机优化问题。该方法可以粗略地看作是 SVRG 算法和新 BB 步长机制的混合体。在目标函数为强凸性的条件下,我们给出了该算法的线性收敛证明。数值实验结果表明,如果算法参数选择得当,SVRG-新 BB 算法的性能可以超越其他现有算法。
本文章由计算机程序翻译,如有差异,请以英文原文为准。

A stochastic variance reduced gradient method with adaptive step for stochastic optimization

A stochastic variance reduced gradient method with adaptive step for stochastic optimization
In this paper, we propose a stochastic variance reduction gradient method with adaptive step size, referred to as the SVRG-New BB method, to solve the convex stochastic optimization problem. The method could be roughly viewed as a hybrid of the SVRG algorithm and a new BB step mechanism. Under the condition that the objective function is strongly convex, we provide the linear convergence proof of this algorithm. Numerical experiment results show that the performance of the SVRG-New BB algorithm can surpass other existing algorithms if parameters in the algorithm are properly chosen.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信