Jeannie He;Ming Xiao;Mikael Skoglund;Harold Vincent Poor
{"title":"Straggler-Resilient Asynchronous ADMM for Distributed Consensus Optimization","authors":"Jeannie He;Ming Xiao;Mikael Skoglund;Harold Vincent Poor","doi":"10.1109/TSP.2025.3579628","DOIUrl":null,"url":null,"abstract":"For its simplicity, well-established convergence properties, and applicability to various optimization problems, the alternating direction method of multipliers (ADMM) has been widely used in several fields. However, when applied in distributed systems, the method may encounter the challenges of stragglers (nodes with significantly longer response time than others) and single points of failure (a single node causing the failure of the entire system). To address these problems, we propose three straggler-resilient ADMM algorithms. The first one is a centralized straggler-resilient ADMM algorithm achieving straggler-resilience by allowing the nodes to proceed to the next iteration even when one or more nodes have not provided an update for one or more iterations. The second one is an extension of the first one achieving single-point-of-failure resilience and fast convergence through decentralized, asynchronous, and concurrent operations. The third one is an extension of the second one to also achieve robustness against uncertainties with the help of a time-tracking scheme. Through theoretical analyses, we establish the convergence properties of our algorithms and show that our algorithms achieve a computational complexity of <inline-formula><tex-math>$\\mathcal{O}(1)$</tex-math></inline-formula> for each worker node - excluding the central node in the centralized algorithm, where the workload complexity is <inline-formula><tex-math>$\\mathcal{O}(N)$</tex-math></inline-formula>. By numerical simulations with various settings, we show that our algorithms have converged significantly faster than several state-of-the-art ADMM algorithms with well-established convergence properties.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"2496-2510"},"PeriodicalIF":4.6000,"publicationDate":"2025-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Signal Processing","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/11037351/","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0
Abstract
For its simplicity, well-established convergence properties, and applicability to various optimization problems, the alternating direction method of multipliers (ADMM) has been widely used in several fields. However, when applied in distributed systems, the method may encounter the challenges of stragglers (nodes with significantly longer response time than others) and single points of failure (a single node causing the failure of the entire system). To address these problems, we propose three straggler-resilient ADMM algorithms. The first one is a centralized straggler-resilient ADMM algorithm achieving straggler-resilience by allowing the nodes to proceed to the next iteration even when one or more nodes have not provided an update for one or more iterations. The second one is an extension of the first one achieving single-point-of-failure resilience and fast convergence through decentralized, asynchronous, and concurrent operations. The third one is an extension of the second one to also achieve robustness against uncertainties with the help of a time-tracking scheme. Through theoretical analyses, we establish the convergence properties of our algorithms and show that our algorithms achieve a computational complexity of $\mathcal{O}(1)$ for each worker node - excluding the central node in the centralized algorithm, where the workload complexity is $\mathcal{O}(N)$. By numerical simulations with various settings, we show that our algorithms have converged significantly faster than several state-of-the-art ADMM algorithms with well-established convergence properties.
期刊介绍:
The IEEE Transactions on Signal Processing covers novel theory, algorithms, performance analyses and applications of techniques for the processing, understanding, learning, retrieval, mining, and extraction of information from signals. The term “signal” includes, among others, audio, video, speech, image, communication, geophysical, sonar, radar, medical and musical signals. Examples of topics of interest include, but are not limited to, information processing and the theory and application of filtering, coding, transmitting, estimating, detecting, analyzing, recognizing, synthesizing, recording, and reproducing signals.