利用深度学习加速前向-后向优化

IF 2.6 1区数学 Q1 MATHEMATICS, APPLIED

SIAM Journal on Optimization Pub Date : 2024-04-01 DOI:10.1137/22m1532548

Sebastian Banert, Jevgenija Rudzusika, Ozan Öktem, Jonas Adler

{"title":"利用深度学习加速前向-后向优化","authors":"Sebastian Banert, Jevgenija Rudzusika, Ozan Öktem, Jonas Adler","doi":"10.1137/22m1532548","DOIUrl":null,"url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 2, Page 1236-1263, June 2024. <br/> Abstract. We propose several deep-learning accelerated optimization solvers with convergence guarantees. We use ideas from the analysis of accelerated forward-backward schemes like FISTA, but instead of the classical approach of proving convergence for a choice of parameters, such as a step-size, we show convergence whenever the update is chosen in a specific set. Rather than picking a point in this set using some predefined method, we train a deep neural network to pick the best update within a given space. Finally, we show that the method is applicable to several cases of smooth and nonsmooth optimization and show superior results to established accelerated solvers.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":"31 1","pages":""},"PeriodicalIF":2.6000,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Accelerated Forward-Backward Optimization Using Deep Learning\",\"authors\":\"Sebastian Banert, Jevgenija Rudzusika, Ozan Öktem, Jonas Adler\",\"doi\":\"10.1137/22m1532548\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"SIAM Journal on Optimization, Volume 34, Issue 2, Page 1236-1263, June 2024. <br/> Abstract. We propose several deep-learning accelerated optimization solvers with convergence guarantees. We use ideas from the analysis of accelerated forward-backward schemes like FISTA, but instead of the classical approach of proving convergence for a choice of parameters, such as a step-size, we show convergence whenever the update is chosen in a specific set. Rather than picking a point in this set using some predefined method, we train a deep neural network to pick the best update within a given space. Finally, we show that the method is applicable to several cases of smooth and nonsmooth optimization and show superior results to established accelerated solvers.\",\"PeriodicalId\":49529,\"journal\":{\"name\":\"SIAM Journal on Optimization\",\"volume\":\"31 1\",\"pages\":\"\"},\"PeriodicalIF\":2.6000,\"publicationDate\":\"2024-04-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"SIAM Journal on Optimization\",\"FirstCategoryId\":\"100\",\"ListUrlMain\":\"https://doi.org/10.1137/22m1532548\",\"RegionNum\":1,\"RegionCategory\":\"数学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"MATHEMATICS, APPLIED\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"SIAM Journal on Optimization","FirstCategoryId":"100","ListUrlMain":"https://doi.org/10.1137/22m1532548","RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MATHEMATICS, APPLIED","Score":null,"Total":0}

引用次数: 0

摘要

SIAM 优化期刊》，第 34 卷第 2 期，第 1236-1263 页，2024 年 6 月。摘要我们提出了几种具有收敛性保证的深度学习加速优化求解器。我们使用了对 FISTA 等加速前向后向方案的分析思路，但我们并没有采用经典的方法来证明步长等参数选择的收敛性，而是证明了在特定集合中选择更新时的收敛性。我们不是使用某种预定义的方法在这个集合中选取一个点，而是训练一个深度神经网络，在给定的空间内选取最佳更新。最后，我们证明该方法适用于平滑和非平滑优化的几种情况，并显示出优于现有加速求解器的结果。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Accelerated Forward-Backward Optimization Using Deep Learning

SIAM Journal on Optimization, Volume 34, Issue 2, Page 1236-1263, June 2024.
Abstract. We propose several deep-learning accelerated optimization solvers with convergence guarantees. We use ideas from the analysis of accelerated forward-backward schemes like FISTA, but instead of the classical approach of proving convergence for a choice of parameters, such as a step-size, we show convergence whenever the update is chosen in a specific set. Rather than picking a point in this set using some predefined method, we train a deep neural network to pick the best update within a given space. Finally, we show that the method is applicable to several cases of smooth and nonsmooth optimization and show superior results to established accelerated solvers.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

SIAM Journal on Optimization 数学-应用数学

CiteScore

5.30

自引率

9.70%

发文量

101

审稿时长

6-12 weeks

期刊介绍： The SIAM Journal on Optimization contains research articles on the theory and practice of optimization. The areas addressed include linear and quadratic programming, convex programming, nonlinear programming, complementarity problems, stochastic optimization, combinatorial optimization, integer programming, and convex, nonsmooth and variational analysis. Contributions may emphasize optimization theory, algorithms, software, computational practice, applications, or the links between these subjects.