云负载均衡器需要远离数据路径

IF 5 2区 计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS
Yuchen Zhang;Shuai Jin;Zhenyu Wen;Shibo He;Qingzheng Hou;Yang Song;Zhigang Zong;Xiaomin Wu;Bengbeng Xue;Chenghao Sun;Ku Li;Xing Li;Biao Lyu;Rong Wen;Jiming Chen;Shunmin Zhu
{"title":"云负载均衡器需要远离数据路径","authors":"Yuchen Zhang;Shuai Jin;Zhenyu Wen;Shibo He;Qingzheng Hou;Yang Song;Zhigang Zong;Xiaomin Wu;Bengbeng Xue;Chenghao Sun;Ku Li;Xing Li;Biao Lyu;Rong Wen;Jiming Chen;Shunmin Zhu","doi":"10.1109/TCC.2025.3595172","DOIUrl":null,"url":null,"abstract":"Load balancers (LBs) are crucial in cloud environments, ensuring workload scalability. They route packets destined for a service (identified by a virtual IP address, or VIP) to a group of servers designated to deliver that service, each with its direct IP address (DIP). Consequently, LBs significantly impact the performance of cloud services and the experience of tenants. Many academic studies focus on specific issues such as designing new load balancing algorithms and developing hardware load balancing devices to enhance the LB’s performance, reliability, and scalability. However, we believe this approach is not ideal for cloud data centers for the following reasons: (i) the increasing demands of users and the variety of cloud service types turn the LB into a bottleneck; and (ii) continually adding machines or upgrading hardware devices can incur substantial costs. In this paper, we propose the Next Generation Load Balancer (NGLB), designed to bypass the TCP connection datapath from the LB, thereby eliminating latency overheads and scalability bottlenecks of traditional cloud LBs. The LB only participates in the TCP connection establishment phase. The three key features of our design are: (i) the introduction of an <italic>active address learning</i> model to redirect traffic and bypass the LB, (ii) a <italic>multi-tenant isolation</i> mechanism for deployment within multi-tenant Virtual Private Cloud networks, and (iii) a distributed flow control method, known as <italic>hierarchical connection cleaner</i>, designed to ensure the availability of backend resources. The evaluation results demonstrate that NGLB reduces latency by 16% and increases nearly 3× throughput. With the same LB resources, NGLB improves 10× rate of new connection establishment. More importantly, five years of operational experience has proven NGLB’s stability for high-bandwidth services.","PeriodicalId":13202,"journal":{"name":"IEEE Transactions on Cloud Computing","volume":"13 3","pages":"1078-1090"},"PeriodicalIF":5.0000,"publicationDate":"2025-08-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Cloud Load Balancers Need to Stay Off the Data Path\",\"authors\":\"Yuchen Zhang;Shuai Jin;Zhenyu Wen;Shibo He;Qingzheng Hou;Yang Song;Zhigang Zong;Xiaomin Wu;Bengbeng Xue;Chenghao Sun;Ku Li;Xing Li;Biao Lyu;Rong Wen;Jiming Chen;Shunmin Zhu\",\"doi\":\"10.1109/TCC.2025.3595172\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Load balancers (LBs) are crucial in cloud environments, ensuring workload scalability. They route packets destined for a service (identified by a virtual IP address, or VIP) to a group of servers designated to deliver that service, each with its direct IP address (DIP). Consequently, LBs significantly impact the performance of cloud services and the experience of tenants. Many academic studies focus on specific issues such as designing new load balancing algorithms and developing hardware load balancing devices to enhance the LB’s performance, reliability, and scalability. However, we believe this approach is not ideal for cloud data centers for the following reasons: (i) the increasing demands of users and the variety of cloud service types turn the LB into a bottleneck; and (ii) continually adding machines or upgrading hardware devices can incur substantial costs. In this paper, we propose the Next Generation Load Balancer (NGLB), designed to bypass the TCP connection datapath from the LB, thereby eliminating latency overheads and scalability bottlenecks of traditional cloud LBs. The LB only participates in the TCP connection establishment phase. The three key features of our design are: (i) the introduction of an <italic>active address learning</i> model to redirect traffic and bypass the LB, (ii) a <italic>multi-tenant isolation</i> mechanism for deployment within multi-tenant Virtual Private Cloud networks, and (iii) a distributed flow control method, known as <italic>hierarchical connection cleaner</i>, designed to ensure the availability of backend resources. The evaluation results demonstrate that NGLB reduces latency by 16% and increases nearly 3× throughput. With the same LB resources, NGLB improves 10× rate of new connection establishment. More importantly, five years of operational experience has proven NGLB’s stability for high-bandwidth services.\",\"PeriodicalId\":13202,\"journal\":{\"name\":\"IEEE Transactions on Cloud Computing\",\"volume\":\"13 3\",\"pages\":\"1078-1090\"},\"PeriodicalIF\":5.0000,\"publicationDate\":\"2025-08-04\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Cloud Computing\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/11108264/\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Cloud Computing","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/11108264/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

摘要

负载平衡器(LBs)在云环境中至关重要,它确保了工作负载的可伸缩性。它们将用于服务(由虚拟IP地址或VIP标识)的数据包路由到指定提供该服务的一组服务器,每个服务器都有其直接IP地址(DIP)。因此,位置服务显著影响云服务的性能和租户的体验。许多学术研究集中在设计新的负载均衡算法和开发硬件负载均衡设备等具体问题上,以提高负载均衡的性能、可靠性和可扩展性。然而,我们认为这种方法对于云数据中心来说并不理想,原因如下:(i)用户需求的增加和云服务类型的多样化使负载均衡成为瓶颈;(ii)不断增加机器或升级硬件设备可能会产生大量成本。在本文中,我们提出了下一代负载均衡器(NGLB),旨在绕过负载均衡器的TCP连接数据路径,从而消除传统云负载均衡器的延迟开销和可扩展性瓶颈。LB只参与TCP连接建立阶段。我们设计的三个关键特征是:(i)引入主动地址学习模型来重定向流量并绕过LB, (ii)在多租户虚拟私有云网络中部署的多租户隔离机制,以及(iii)分布式流量控制方法,称为分层连接清理器,旨在确保后端资源的可用性。评估结果表明,NGLB减少了16%的延迟,提高了近3倍的吞吐量。在LB资源相同的情况下,NGLB的新连接建立率提高了10倍。更重要的是,5年的运行经验证明了NGLB在高带宽服务中的稳定性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Cloud Load Balancers Need to Stay Off the Data Path
Load balancers (LBs) are crucial in cloud environments, ensuring workload scalability. They route packets destined for a service (identified by a virtual IP address, or VIP) to a group of servers designated to deliver that service, each with its direct IP address (DIP). Consequently, LBs significantly impact the performance of cloud services and the experience of tenants. Many academic studies focus on specific issues such as designing new load balancing algorithms and developing hardware load balancing devices to enhance the LB’s performance, reliability, and scalability. However, we believe this approach is not ideal for cloud data centers for the following reasons: (i) the increasing demands of users and the variety of cloud service types turn the LB into a bottleneck; and (ii) continually adding machines or upgrading hardware devices can incur substantial costs. In this paper, we propose the Next Generation Load Balancer (NGLB), designed to bypass the TCP connection datapath from the LB, thereby eliminating latency overheads and scalability bottlenecks of traditional cloud LBs. The LB only participates in the TCP connection establishment phase. The three key features of our design are: (i) the introduction of an active address learning model to redirect traffic and bypass the LB, (ii) a multi-tenant isolation mechanism for deployment within multi-tenant Virtual Private Cloud networks, and (iii) a distributed flow control method, known as hierarchical connection cleaner, designed to ensure the availability of backend resources. The evaluation results demonstrate that NGLB reduces latency by 16% and increases nearly 3× throughput. With the same LB resources, NGLB improves 10× rate of new connection establishment. More importantly, five years of operational experience has proven NGLB’s stability for high-bandwidth services.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
IEEE Transactions on Cloud Computing
IEEE Transactions on Cloud Computing Computer Science-Software
CiteScore
9.40
自引率
6.20%
发文量
167
期刊介绍: The IEEE Transactions on Cloud Computing (TCC) is dedicated to the multidisciplinary field of cloud computing. It is committed to the publication of articles that present innovative research ideas, application results, and case studies in cloud computing, focusing on key technical issues related to theory, algorithms, systems, applications, and performance.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信