Optimization Methods and Software最新文献_第6页

On the numerical performance of finite-difference-based methods for derivative-free optimization 基于有限差分的无导数优化方法的数值性能研究

Optimization Methods and Software Pub Date : 2022-09-26 DOI: 10.1080/10556788.2022.2121832

H. Shi, M. Xuan, Figen Öztoprak, J. Nocedal

{"title":"On the numerical performance of finite-difference-based methods for derivative-free optimization","authors":"H. Shi, M. Xuan, Figen Öztoprak, J. Nocedal","doi":"10.1080/10556788.2022.2121832","DOIUrl":"https://doi.org/10.1080/10556788.2022.2121832","url":null,"abstract":"The goal of this paper is to investigate an approach for derivative-free optimization that has not received sufficient attention in the literature and is yet one of the simplest to implement and parallelize. In its simplest form, it consists of employing derivative-based methods for unconstrained or constrained optimization and replacing the gradient of the objective (and constraints) by finite-difference approximations. This approach is applicable to problems with or without noise in the functions. The differencing interval is determined by a bound on the second (or third) derivative and by the noise level, which is assumed to be known or to be accessible through difference tables or sampling. The use of finite-difference gradient approximations has been largely dismissed in the derivative-free optimization literature as too expensive in terms of function evaluations or as impractical in the presence of noise. However, the test results presented in this paper suggest that it has much to be recommended. The experiments compare newuoa, dfo-ls and cobyla against finite-difference versions of l-bfgs, lmder and knitro on three classes of problems: general unconstrained problems, nonlinear least squares problems and nonlinear programs with inequality constraints.","PeriodicalId":124811,"journal":{"name":"Optimization Methods and Software","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128557413","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

A new randomized primal-dual algorithm for convex optimization with fast last iterate convergence rates 一种新的具有快速迭代收敛速度的随机原对偶凸优化算法

Optimization Methods and Software Pub Date : 2022-09-26 DOI: 10.1080/10556788.2022.2119233

Quoc Tran-Dinh, Deyi Liu

{"title":"A new randomized primal-dual algorithm for convex optimization with fast last iterate convergence rates","authors":"Quoc Tran-Dinh, Deyi Liu","doi":"10.1080/10556788.2022.2119233","DOIUrl":"https://doi.org/10.1080/10556788.2022.2119233","url":null,"abstract":"We develop a novel unified randomized block-coordinate primal-dual algorithm to solve a class of nonsmooth constrained convex optimization problems, which covers different existing variants and model settings from the literature. We prove that our algorithm achieves and convergence rates in two cases: merely convexity and strong convexity, respectively, where k is the iteration counter and n is the number of block-coordinates. These rates are known to be optimal (up to a constant factor) when n = 1. Our convergence rates are obtained through three criteria: primal objective residual and primal feasibility violation, dual objective residual, and primal-dual expected gap. Moreover, our rates for the primal problem are on the last-iterate sequence. Our dual convergence guarantee requires additionally a Lipschitz continuity assumption. We specify our algorithm to handle two important special cases, where our rates are still applied. Finally, we verify our algorithm on two well-studied numerical examples and compare it with two existing methods. Our results show that the proposed method has encouraging performance on different experiments.","PeriodicalId":124811,"journal":{"name":"Optimization Methods and Software","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129239659","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

New iterative algorithms with self-adaptive step size for solving split equality fixed point problem and its applications 解分裂等式不动点问题的自适应步长迭代算法及其应用

Optimization Methods and Software Pub Date : 2022-09-09 DOI: 10.1080/10556788.2022.2117357

Yan Tang, Haiyun Zhou

引用次数: 0

Practical perspectives on symplectic accelerated optimization 辛加速优化的实用观点

Optimization Methods and Software Pub Date : 2022-07-23 DOI: 10.1080/10556788.2023.2214837

Valentin Duruisseaux, M. Leok

{"title":"Practical perspectives on symplectic accelerated optimization","authors":"Valentin Duruisseaux, M. Leok","doi":"10.1080/10556788.2023.2214837","DOIUrl":"https://doi.org/10.1080/10556788.2023.2214837","url":null,"abstract":"Geometric numerical integration has recently been exploited to design symplectic accelerated optimization algorithms by simulating the Lagrangian and Hamiltonian systems from the variational framework introduced in Wibisono et al. In this paper, we discuss practical considerations which can significantly boost the computational performance of these optimization algorithms, and considerably simplify the tuning process. In particular, we investigate how momentum restarting schemes ameliorate computational efficiency and robustness by reducing the undesirable effect of oscillations, and ease the tuning process by making time-adaptivity superfluous. We also discuss how temporal looping helps avoiding instability issues caused by numerical precision, without harming the computational efficiency of the algorithms. Finally, we compare the efficiency and robustness of different geometric integration techniques, and study the effects of the different parameters in the algorithms to inform and simplify tuning in practice. From this paper emerge symplectic accelerated optimization algorithms whose computational efficiency, stability and robustness have been improved, and which are now much simpler to use and tune for practical applications.","PeriodicalId":124811,"journal":{"name":"Optimization Methods and Software","volume":"45 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127179302","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Exact gradient methods with memory 具有记忆的精确梯度方法

Optimization Methods and Software Pub Date : 2022-07-20 DOI: 10.1080/10556788.2022.2091559

Mihai I. Florea

引用次数: 2

Sequential approximate optimization with adaptive parallel infill strategy assisted by inaccurate Pareto front 基于不准确Pareto前沿的自适应并行填充策略序列逼近优化

Optimization Methods and Software Pub Date : 2022-07-19 DOI: 10.1080/10556788.2022.2091560

Wenjie Wang, Pengyu Wang, Jiawei Yang, Fei Xiao, Weihua Zhang, Zeping Wu

引用次数: 0

A stochastic approximation method for convex programming with many semidefinite constraints 多半定约束凸规划的随机逼近方法

Optimization Methods and Software Pub Date : 2022-07-19 DOI: 10.1080/10556788.2022.2091563

L. Pang, Ming-Kun Zhang, X. Xiao

引用次数: 0

On inexact stochastic splitting methods for a class of nonconvex composite optimization problems with relative error 一类具有相对误差的非凸复合优化问题的非精确随机分裂方法

Optimization Methods and Software Pub Date : 2022-07-19 DOI: 10.1080/10556788.2022.2091562

Jia Hu, Congying Han, Tiande Guo, Tong Zhao

{"title":"On inexact stochastic splitting methods for a class of nonconvex composite optimization problems with relative error","authors":"Jia Hu, Congying Han, Tiande Guo, Tong Zhao","doi":"10.1080/10556788.2022.2091562","DOIUrl":"https://doi.org/10.1080/10556788.2022.2091562","url":null,"abstract":"We consider minimizing a class of nonconvex composite stochastic optimization problems, and deterministic optimization problems whose objective function consists of an expectation function (or an average of finitely many smooth functions) and a weakly convex but potentially nonsmooth function. And in this paper, we focus on the theoretical properties of two types of stochastic splitting methods for solving these nonconvex optimization problems: stochastic alternating direction method of multipliers and stochastic proximal gradient descent. In particular, several inexact versions of these two types of splitting methods are studied. At each iteration, the proposed schemes inexactly solve their subproblems by using relative error criteria instead of exogenous and diminishing error rules, which allows our approaches to handle some complex regularized problems in statistics and machine learning. Under mild conditions, we obtain the convergence of the schemes and their computational complexity related to the evaluations on the component gradient of the smooth function, and find that some conclusions of their exact counterparts can be recovered.","PeriodicalId":124811,"journal":{"name":"Optimization Methods and Software","volume":"122 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-07-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116841254","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A quasi-Newton method in shape optimization for a transmission problem 传动问题形状优化的准牛顿方法

Optimization Methods and Software Pub Date : 2022-06-01 DOI: 10.1080/10556788.2022.2078823

Petar Kunštek, M. Vrdoljak

引用次数: 0

A wide neighbourhood predictor–corrector infeasible-interior-point algorithm for symmetric cone programming 对称锥规划的宽邻域预测校正不可行内点算法

Optimization Methods and Software Pub Date : 2022-05-12 DOI: 10.1080/10556788.2022.2060970

M. S. Shahraki, H. Mansouri, A. Delavarkhalafi

引用次数: 0