LRCC: Long-haul RDMA congestion control for cross-datacenter networks

Abstract

With the widespread deployment of applications such as cloud storage and distributed model training, Remote Direct Memory Access (RDMA) is increasingly applied to cross-datacenter networks. These networks typically consist of multiple regional datacenters interconnected by dedicated long-haul optical fiber and Data Center Interconnect (DCI) switches. However, existing RDMA congestion control mechanisms face significant challenges in cross-datacenter networks. Firstly, the long control loops struggle to effectively suppress line-rate bursts of cross-domain RDMA traffic, leading to persistent queues that degrade overall network performance. Secondly, the heterogeneous Round-Trip Time (RTT) characteristics between cross-domain and intra-datacenter traffic disrupt the convergence and fairness guarantees of conventional methods, further exacerbating cross-domain congestion issues. In this paper, we propose a switch-driven Long-haul RDMA Congestion Control (LRCC). LRCC utilizes near-source switches to generate congestion notification packets, effectively shortening the long control loops. Furthermore, LRCC implements a precise fair-rate computation mechanism on the switches and an adaptive rate-increase strategy on the host. These mechanisms mitigate cross-domain congestion caused by hybrid traffic while ensuring high throughput for long-haul flows. We implemented a prototype system of LRCC on programmable switches and 400Gbps FPGA NICs. Testbed experiments show that, compared with the NVIDIA CX7, LRCC reduces tail latency by 11 %-16 % in short-distance congestion scenarios and by 45 %-49 % in a 640 km long-distance scenario. Large-scale simulations further demonstrate that in the cross-datacenter networks, LRCC outperforms existing solutions, reducing the average Flow Completion Time (FCT) by up to 67.2 %, 94 % and 48.4 %, respectively, compared to DCQCN, HPCC and BiCC.