OWL: Worker-assisted server bandwidth optimization for efficient communication federated learning

IF 3.4 3区 计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS
Xiaoming Han , Boan Liu , Chuang Hu , Dazhao Cheng
{"title":"OWL: Worker-assisted server bandwidth optimization for efficient communication federated learning","authors":"Xiaoming Han ,&nbsp;Boan Liu ,&nbsp;Chuang Hu ,&nbsp;Dazhao Cheng","doi":"10.1016/j.jpdc.2024.105027","DOIUrl":null,"url":null,"abstract":"<div><div>Edge computing in federated learning based on centralized architecture often faces communication constraints in large clusters. Although there have been some efforts like computation-communication overlapping and fine-granularity flow scheduling towards how to reduce the communication cost, this is still a matter of ongoing research. Motivated by the underutilization of bandwidth among workers (edge devices) and the replication of deep neural network (DNN) model distributions in data-parallel federated learning, we propose OWL, a novel worker-assisted server bandwidth optimization method. OWL partitions numerous computation branches into groups based on the model's network topology, allowing for overlapping model distribution and computation among workers, thereby leveraging idle communication resources on the workers to compensate for server bandwidth. To address the issue of model distribution congestion on the server, we formulate group partition as an optimization problem, which proves to be NP-hard. We tackle this problem through a divide-and-conquer approach employing an approximation grouping algorithm and a deploying algorithm. Finally, we evaluate the performance of OWL through simulations and a comprehensive real-world case study involving model training on OWL and deployment on edge systems. Experimental results demonstrate that OWL reduces overall training time by up to 20%-69% and improves scalability by over 9.5% compared to state-of-the-art overlapping approaches.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"198 ","pages":"Article 105027"},"PeriodicalIF":3.4000,"publicationDate":"2024-12-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Parallel and Distributed Computing","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0743731524001916","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, THEORY & METHODS","Score":null,"Total":0}
引用次数: 0

Abstract

Edge computing in federated learning based on centralized architecture often faces communication constraints in large clusters. Although there have been some efforts like computation-communication overlapping and fine-granularity flow scheduling towards how to reduce the communication cost, this is still a matter of ongoing research. Motivated by the underutilization of bandwidth among workers (edge devices) and the replication of deep neural network (DNN) model distributions in data-parallel federated learning, we propose OWL, a novel worker-assisted server bandwidth optimization method. OWL partitions numerous computation branches into groups based on the model's network topology, allowing for overlapping model distribution and computation among workers, thereby leveraging idle communication resources on the workers to compensate for server bandwidth. To address the issue of model distribution congestion on the server, we formulate group partition as an optimization problem, which proves to be NP-hard. We tackle this problem through a divide-and-conquer approach employing an approximation grouping algorithm and a deploying algorithm. Finally, we evaluate the performance of OWL through simulations and a comprehensive real-world case study involving model training on OWL and deployment on edge systems. Experimental results demonstrate that OWL reduces overall training time by up to 20%-69% and improves scalability by over 9.5% compared to state-of-the-art overlapping approaches.
OWL:工作人员辅助的服务器带宽优化,用于有效的通信联合学习
基于集中式体系结构的联邦学习中的边缘计算在大型集群中经常面临通信约束。虽然在如何降低通信成本方面已经有了一些努力,如计算通信重叠和细粒度流调度,但这仍然是一个正在进行的研究问题。针对工作人员(边缘设备)之间的带宽利用率不足以及数据并行联邦学习中深度神经网络(DNN)模型分布的复制,我们提出了一种新的工作人员辅助服务器带宽优化方法OWL。OWL根据模型的网络拓扑将众多计算分支划分为组,允许在worker之间重叠模型分布和计算,从而利用worker上的空闲通信资源来补偿服务器带宽。为了解决服务器上的模型分布拥塞问题,我们将分组划分作为一个优化问题,这被证明是np困难的。我们通过采用近似分组算法和部署算法的分而治之的方法来解决这个问题。最后,我们通过模拟和全面的现实世界案例研究来评估OWL的性能,包括OWL的模型训练和在边缘系统上的部署。实验结果表明,与最先进的重叠方法相比,OWL将总训练时间减少了20%-69%,可扩展性提高了9.5%以上。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Journal of Parallel and Distributed Computing
Journal of Parallel and Distributed Computing 工程技术-计算机:理论方法
CiteScore
10.30
自引率
2.60%
发文量
172
审稿时长
12 months
期刊介绍: This international journal is directed to researchers, engineers, educators, managers, programmers, and users of computers who have particular interests in parallel processing and/or distributed computing. The Journal of Parallel and Distributed Computing publishes original research papers and timely review articles on the theory, design, evaluation, and use of parallel and/or distributed computing systems. The journal also features special issues on these topics; again covering the full range from the design to the use of our targeted systems.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信