{"title":"通过惯性算法的凸优化与消失的 Tikhonov 正则化:快速收敛至最小规范解","authors":"Hedy Attouch, Szilárd Csaba László","doi":"10.1007/s00186-024-00867-y","DOIUrl":null,"url":null,"abstract":"<p>In a Hilbertian framework, for the minimization of a general convex differentiable function <i>f</i>, we introduce new inertial dynamics and algorithms that generate trajectories and iterates that converge fastly towards the minimizer of <i>f</i> with minimum norm. Our study is based on the non-autonomous version of the Polyak heavy ball method, which, at time <i>t</i>, is associated with the strongly convex function obtained by adding to <i>f</i> a Tikhonov regularization term with vanishing coefficient <span>\\(\\varepsilon (t)\\)</span>. In this dynamic, the damping coefficient is proportional to the square root of the Tikhonov regularization parameter <span>\\(\\varepsilon (t)\\)</span>. By adjusting the speed of convergence of <span>\\(\\varepsilon (t)\\)</span> towards zero, we will obtain both rapid convergence towards the infimal value of <i>f</i>, and the strong convergence of the trajectories towards the element of minimum norm of the set of minimizers of <i>f</i>. In particular, we obtain an improved version of the dynamic of Su-Boyd-Candès for the accelerated gradient method of Nesterov. This study naturally leads to corresponding first-order algorithms obtained by temporal discretization. In the case of a proper lower semicontinuous and convex function <i>f</i>, we study the proximal algorithms in detail, and show that they benefit from similar properties.\n</p>","PeriodicalId":49862,"journal":{"name":"Mathematical Methods of Operations Research","volume":"196 1","pages":""},"PeriodicalIF":0.9000,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Convex optimization via inertial algorithms with vanishing Tikhonov regularization: fast convergence to the minimum norm solution\",\"authors\":\"Hedy Attouch, Szilárd Csaba László\",\"doi\":\"10.1007/s00186-024-00867-y\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>In a Hilbertian framework, for the minimization of a general convex differentiable function <i>f</i>, we introduce new inertial dynamics and algorithms that generate trajectories and iterates that converge fastly towards the minimizer of <i>f</i> with minimum norm. Our study is based on the non-autonomous version of the Polyak heavy ball method, which, at time <i>t</i>, is associated with the strongly convex function obtained by adding to <i>f</i> a Tikhonov regularization term with vanishing coefficient <span>\\\\(\\\\varepsilon (t)\\\\)</span>. In this dynamic, the damping coefficient is proportional to the square root of the Tikhonov regularization parameter <span>\\\\(\\\\varepsilon (t)\\\\)</span>. By adjusting the speed of convergence of <span>\\\\(\\\\varepsilon (t)\\\\)</span> towards zero, we will obtain both rapid convergence towards the infimal value of <i>f</i>, and the strong convergence of the trajectories towards the element of minimum norm of the set of minimizers of <i>f</i>. In particular, we obtain an improved version of the dynamic of Su-Boyd-Candès for the accelerated gradient method of Nesterov. This study naturally leads to corresponding first-order algorithms obtained by temporal discretization. In the case of a proper lower semicontinuous and convex function <i>f</i>, we study the proximal algorithms in detail, and show that they benefit from similar properties.\\n</p>\",\"PeriodicalId\":49862,\"journal\":{\"name\":\"Mathematical Methods of Operations Research\",\"volume\":\"196 1\",\"pages\":\"\"},\"PeriodicalIF\":0.9000,\"publicationDate\":\"2024-06-27\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Mathematical Methods of Operations Research\",\"FirstCategoryId\":\"100\",\"ListUrlMain\":\"https://doi.org/10.1007/s00186-024-00867-y\",\"RegionNum\":4,\"RegionCategory\":\"数学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"MATHEMATICS, APPLIED\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Mathematical Methods of Operations Research","FirstCategoryId":"100","ListUrlMain":"https://doi.org/10.1007/s00186-024-00867-y","RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"MATHEMATICS, APPLIED","Score":null,"Total":0}
引用次数: 0
摘要
在希尔伯特框架下,针对一般凸可微函数 f 的最小化问题,我们引入了新的惯性动力学和算法,其生成的轨迹和迭代可快速收敛至 f 的最小化且具有最小规范。我们的研究基于非自主版本的波利克重球方法,在时间 t 上,该方法与强凸函数相关联,强凸函数是通过在 f 上添加一个具有消失系数 \(\varepsilon (t)\) 的 Tikhonov 正则化项而得到的。在这种动态中,阻尼系数与 Tikhonov 正则化参数 \(\varepsilon (t)\) 的平方根成正比。通过将 \(\varepsilon (t)\) 的收敛速度调整为零,我们将同时获得向 f 的次极值快速收敛和向 f 最小化集合的最小规范元素强收敛的轨迹。这项研究自然会引出通过时间离散化获得的相应一阶算法。在适当的下半连续凸函数 f 的情况下,我们详细研究了近似算法,并证明它们受益于类似的性质。
Convex optimization via inertial algorithms with vanishing Tikhonov regularization: fast convergence to the minimum norm solution
In a Hilbertian framework, for the minimization of a general convex differentiable function f, we introduce new inertial dynamics and algorithms that generate trajectories and iterates that converge fastly towards the minimizer of f with minimum norm. Our study is based on the non-autonomous version of the Polyak heavy ball method, which, at time t, is associated with the strongly convex function obtained by adding to f a Tikhonov regularization term with vanishing coefficient \(\varepsilon (t)\). In this dynamic, the damping coefficient is proportional to the square root of the Tikhonov regularization parameter \(\varepsilon (t)\). By adjusting the speed of convergence of \(\varepsilon (t)\) towards zero, we will obtain both rapid convergence towards the infimal value of f, and the strong convergence of the trajectories towards the element of minimum norm of the set of minimizers of f. In particular, we obtain an improved version of the dynamic of Su-Boyd-Candès for the accelerated gradient method of Nesterov. This study naturally leads to corresponding first-order algorithms obtained by temporal discretization. In the case of a proper lower semicontinuous and convex function f, we study the proximal algorithms in detail, and show that they benefit from similar properties.
期刊介绍:
This peer reviewed journal publishes original and high-quality articles on important mathematical and computational aspects of operations research, in particular in the areas of continuous and discrete mathematical optimization, stochastics, and game theory. Theoretically oriented papers are supposed to include explicit motivations of assumptions and results, while application oriented papers need to contain substantial mathematical contributions. Suggestions for algorithms should be accompanied with numerical evidence for their superiority over state-of-the-art methods. Articles must be of interest for a large audience in operations research, written in clear and correct English, and typeset in LaTeX. A special section contains invited tutorial papers on advanced mathematical or computational aspects of operations research, aiming at making such methodologies accessible for a wider audience.
All papers are refereed. The emphasis is on originality, quality, and importance.