Randomized Douglas–Rachford Methods for Linear Systems: Improved Accuracy and Efficiency

IF 2.6 1区数学 Q1 MATHEMATICS, APPLIED

SIAM Journal on Optimization Pub Date : 2024-03-15 DOI:10.1137/23m1567503

Deren Han, Yansheng Su, Jiaxin Xie

{"title":"Randomized Douglas–Rachford Methods for Linear Systems: Improved Accuracy and Efficiency","authors":"Deren Han, Yansheng Su, Jiaxin Xie","doi":"10.1137/23m1567503","DOIUrl":null,"url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 1, Page 1045-1070, March 2024. <br/> Abstract. The Douglas–Rachford (DR) method is a widely used method for finding a point in the intersection of two closed convex sets (feasibility problem). However, the method converges weakly, and the associated rate of convergence is hard to analyze in general. In addition, the direct extension of the DR method for solving more-than-two-sets feasibility problems, called the [math]-sets-DR method, is not necessarily convergent. To improve the efficiency of the optimization algorithms, the introduction of randomization and the momentum technique has attracted increasing attention. In this paper, we propose the randomized [math]-sets-DR (RrDR) method for solving the feasibility problem derived from linear systems, showing the benefit of the randomization as it brings linear convergence in expectation to the otherwise divergent [math]-sets-DR method. Furthermore, the convergence rate does not depend on the dimension of the coefficient matrix. We also study RrDR with heavy ball momentum and establish its accelerated rate. Numerical experiments are provided to confirm our results and demonstrate the notable improvements in accuracy and efficiency of the DR method brought by the randomization and the momentum technique.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":2.6000,"publicationDate":"2024-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"SIAM Journal on Optimization","FirstCategoryId":"100","ListUrlMain":"https://doi.org/10.1137/23m1567503","RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MATHEMATICS, APPLIED","Score":null,"Total":0}

引用次数: 0

Abstract

SIAM Journal on Optimization, Volume 34, Issue 1, Page 1045-1070, March 2024.
Abstract. The Douglas–Rachford (DR) method is a widely used method for finding a point in the intersection of two closed convex sets (feasibility problem). However, the method converges weakly, and the associated rate of convergence is hard to analyze in general. In addition, the direct extension of the DR method for solving more-than-two-sets feasibility problems, called the [math]-sets-DR method, is not necessarily convergent. To improve the efficiency of the optimization algorithms, the introduction of randomization and the momentum technique has attracted increasing attention. In this paper, we propose the randomized [math]-sets-DR (RrDR) method for solving the feasibility problem derived from linear systems, showing the benefit of the randomization as it brings linear convergence in expectation to the otherwise divergent [math]-sets-DR method. Furthermore, the convergence rate does not depend on the dimension of the coefficient matrix. We also study RrDR with heavy ball momentum and establish its accelerated rate. Numerical experiments are provided to confirm our results and demonstrate the notable improvements in accuracy and efficiency of the DR method brought by the randomization and the momentum technique.

查看原文本刊更多论文

线性系统的随机化道格拉斯-拉赫福德方法：提高精度和效率

SIAM 优化期刊》，第 34 卷，第 1 期，第 1045-1070 页，2024 年 3 月。摘要道格拉斯-拉克福德（Douglas-Rachford，DR）方法是一种广泛应用于寻找两个闭合凸集交点（可行性问题）的方法。然而，该方法的收敛性较弱，相关的收敛速率一般难以分析。此外，DR 方法的直接扩展用于求解多于两个集合的可行性问题，即[math]-sets-DR 方法，也不一定收敛。为了提高优化算法的效率，随机化和动量技术的引入引起了越来越多的关注。本文提出了随机化[math]-sets-DR（RrDR）方法，用于求解线性系统衍生的可行性问题，显示了随机化的好处，因为它给原本发散的[math]-sets-DR方法带来了期望值上的线性收敛。此外，收敛速度并不取决于系数矩阵的维度。我们还研究了重球动量下的 RrDR，并确定了其加速率。我们提供了数值实验来证实我们的结果，并证明随机化和动量技术显著提高了 DR 方法的精度和效率。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

SIAM Journal on Optimization 数学-应用数学

CiteScore

5.30

自引率

9.70%

发文量

101

审稿时长

6-12 weeks

期刊介绍： The SIAM Journal on Optimization contains research articles on the theory and practice of optimization. The areas addressed include linear and quadratic programming, convex programming, nonlinear programming, complementarity problems, stochastic optimization, combinatorial optimization, integer programming, and convex, nonsmooth and variational analysis. Contributions may emphasize optimization theory, algorithms, software, computational practice, applications, or the links between these subjects.