Optimization Letters最新文献_第2页

Improved randomized approaches to the location of a conservative hyperplane 保守超平面位置的改进随机方法

IF 1.6 4区数学

Optimization Letters Pub Date : 2024-07-06 DOI: 10.1007/s11590-024-02136-7

Xiaosong Ding, Jun Ma, Xiuming Li, Xi Chen

引用次数: 0

The modified second APG method for a class of nonconvex nonsmooth problems 一类非凸非光滑问题的修正第二 APG 方法

IF 1.6 4区数学

Optimization Letters Pub Date : 2024-07-06 DOI: 10.1007/s11590-024-02132-x

Kexin Ren, Chunguang Liu, Lumiao Wang

引用次数: 0

A limited memory subspace minimization conjugate gradient algorithm for unconstrained optimization 用于无约束优化的有限内存子空间最小化共轭梯度算法

IF 1.6 4区数学

Optimization Letters Pub Date : 2024-07-02 DOI: 10.1007/s11590-024-02131-y

Zexian Liu, Yu-Hong Dai, Hongwei Liu

{"title":"A limited memory subspace minimization conjugate gradient algorithm for unconstrained optimization","authors":"Zexian Liu, Yu-Hong Dai, Hongwei Liu","doi":"10.1007/s11590-024-02131-y","DOIUrl":"https://doi.org/10.1007/s11590-024-02131-y","url":null,"abstract":"Subspace minimization conjugate gradient (SMCG) methods are a class of quite efficient iterative methods for unconstrained optimization. The orthogonality is an important property of linear conjugate gradient method. It is however observed that the orthogonality of the gradients in linear conjugate gradient method is often lost, which usually causes slow convergence. Based on SMCG(_)BB (Liu and Liu in J Optim Theory Appl 180(3):879–906, 2019), we combine subspace minimization conjugate gradient method with the limited memory technique and present a limited memory subspace minimization conjugate gradient algorithm for unconstrained optimization. The proposed method includes two types of iterations: SMCG iteration and quasi-Newton (QN) iteration. In the SMCG iteration, the search direction is determined by solving a quadratic approximation problem, in which the important parameter is estimated based on some properties of the objective function at the current iterative point. In the QN iteration, a modified quasi-Newton method in the subspace is proposed to improve the orthogonality. Additionally, a modified strategy for choosing the initial stepsize is exploited. The global convergence of the proposed method is established under weak conditions. Some numerical results indicate that, for the tested functions in the CUTEr library, the proposed method has a great improvement over SMCG(_)BB, and it is comparable to the latest limited memory conjugate gradient software package CG(_)DESCENT (6.8) (Hager and Zhang in SIAM J Optim 23(4):2150–2168, 2013) and is also superior to the famous limited memory BFGS (L-BFGS) method.","PeriodicalId":49720,"journal":{"name":"Optimization Letters","volume":"21 1","pages":""},"PeriodicalIF":1.6,"publicationDate":"2024-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141501395","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An accelerated lyapunov function for Polyak’s Heavy-ball on convex quadratics 凸四边形上波利克重球的加速拉普诺夫函数

IF 1.6 4区数学

Optimization Letters Pub Date : 2024-06-25 DOI: 10.1007/s11590-024-02119-8

Antonio Orvieto

{"title":"An accelerated lyapunov function for Polyak’s Heavy-ball on convex quadratics","authors":"Antonio Orvieto","doi":"10.1007/s11590-024-02119-8","DOIUrl":"https://doi.org/10.1007/s11590-024-02119-8","url":null,"abstract":"In 1964, Polyak showed that the Heavy-ball method, the simplest momentum technique, accelerates convergence of strongly-convex problems in the vicinity of the solution. While Nesterov later developed a globally accelerated version, Polyak’s original algorithm remains simpler and more widely used in applications such as deep learning. Despite this popularity, the question of whether Heavy-ball is also globally accelerated or not has not been fully answered yet, and no convincing counterexample has been provided. This is largely due to the difficulty in finding an effective Lyapunov function: indeed, most proofs of Heavy-ball acceleration in the strongly-convex quadratic setting rely on eigenvalue arguments. Our work adopts a different approach: studying momentum through the lens of quadratic invariants of simple harmonic oscillators. By utilizing the modified Hamiltonian of Störmer-Verlet integrators, we are able to construct a Lyapunov function that demonstrates an (O(1/k^2)) rate for Heavy-ball in the case of convex quadratic problems. Our novel proof technique, though restricted to linear regression, is found to work well empirically also on non-quadratic convex problems, and thus provides insights on the structure of Lyapunov functions to be used in the general convex case. As such, our paper makes a promising first step towards potentially proving the acceleration of Polyak’s momentum method and we hope it inspires further research around this question.","PeriodicalId":49720,"journal":{"name":"Optimization Letters","volume":"24 1","pages":""},"PeriodicalIF":1.6,"publicationDate":"2024-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141501227","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Benchmark-based deviation and drawdown measures in portfolio optimization 投资组合优化中基于基准的偏差和缩减测量方法

IF 1.6 4区数学

Optimization Letters Pub Date : 2024-06-25 DOI: 10.1007/s11590-024-02124-x

Michael Zabarankin, Bogdan Grechuk, Dawei Hao

引用次数: 0

Strategy investments in zero-sum games 零和博弈中的战略投资

IF 1.6 4区数学

Optimization Letters Pub Date : 2024-06-24 DOI: 10.1007/s11590-024-02130-z

Raul Garcia, Seyedmohammadhossein Hosseinian, Mallesh Pai, Andrew J. Schaefer

引用次数: 0

On the linear convergence rate of Riemannian proximal gradient method 论黎曼近似梯度法的线性收敛速率

IF 1.6 4区数学

Optimization Letters Pub Date : 2024-06-19 DOI: 10.1007/s11590-024-02129-6

Woocheol Choi, Changbum Chun, Yoon Mo Jung, Sangwoon Yun

{"title":"On the linear convergence rate of Riemannian proximal gradient method","authors":"Woocheol Choi, Changbum Chun, Yoon Mo Jung, Sangwoon Yun","doi":"10.1007/s11590-024-02129-6","DOIUrl":"https://doi.org/10.1007/s11590-024-02129-6","url":null,"abstract":"Composite optimization problems on Riemannian manifolds arise in applications such as sparse principal component analysis and dictionary learning. Recently, Huang and Wei introduced a Riemannian proximal gradient method (Huang and Wei in MP 194:371–413, 2022) and an inexact Riemannian proximal gradient method (Wen and Ke in COA 85:1–32, 2023), utilizing the retraction mapping to address these challenges. They established the sublinear convergence rate of the Riemannian proximal gradient method under the retraction convexity and a geometric condition on retractions, as well as the local linear convergence rate of the inexact Riemannian proximal gradient method under the Riemannian Kurdyka-Lojasiewicz property. In this paper, we demonstrate the linear convergence rate of the Riemannian proximal gradient method and the linear convergence rate of the proximal gradient method proposed in Chen et al. (SIAM J Opt 30:210–239, 2020) under strong retraction convexity. Additionally, we provide a counterexample that violates the geometric condition on retractions, which is crucial for establishing the sublinear convergence rate of the Riemannian proximal gradient method.","PeriodicalId":49720,"journal":{"name":"Optimization Letters","volume":"5 1","pages":""},"PeriodicalIF":1.6,"publicationDate":"2024-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141501230","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A modification of the forward–backward splitting method for monotone inclusions 单调夹杂物前向后分裂法的修正

IF 1.6 4区数学

Optimization Letters Pub Date : 2024-06-18 DOI: 10.1007/s11590-024-02128-7

Van Dung Nguyen

引用次数: 0

Solution polishing via path relinking for continuous black-box optimization 通过路径重链接对连续黑箱优化的解决方案进行打磨

IF 1.6 4区数学

Optimization Letters Pub Date : 2024-06-12 DOI: 10.1007/s11590-024-02127-8

Dimitri J. Papageorgiou, Jan Kronqvist, Asha Ramanujam, James Kor, Youngdae Kim, Can Li

引用次数: 0

Dual and generalized dual cones in Banach spaces 巴拿赫空间中的对偶锥和广义对偶锥