TCNet: cross-node virtual machine communication acceleration

Abstract

Driven by rapid development of cloud computing, virtualized environments are becoming popular in data center. Frequent communication among multiple virtual machines is required by a large amount of applications. Although many virtualization acceleration techniques have been proposed, the network performance is still a hot research topic due to the complicated and costly implementations of I/O virtualization mechanism. Some previous research focuses on improving the efficiency of communication among virtual machines in the same host. But studying how to accelerate cross-node virtual machine communication is also necessary. On the other hand, many high efficient, tight-coupling interconnects have been proposed as data center interconnects. They have advantages in performance and efficiency, while traditional Ethernet and InfiniBand have good scalability. However, these two kinds of interconnects can coexist very well. Tight-coupling protocol is suitable for connecting small-scale data center nodes, which we call super-node, while super-node is connected by traditional interconnect. In our opinion, data center with such hybrid interconnect architecture is one of important trends. Targeting the hybrid interconnect architecture, this paper proposes an efficient mechanism, named as TCNet (abbreviation for tight-coupling network), to accelerate cross-node virtual machine communication. To verify the acceleration mechanism, we build a prototype system which chooses PCIe (for inner-super-node interconnect) and Ethernet (for inter-super-node interconnect) as the hybrid interconnect and use KVM as software environments. We use several benchmarks to evaluate the mechanism. The latency of TCNet is 23% shorter than that of Gigabit Ethernet on average and the bandwidth is 1.14 times as large as that of Gigabit Ethernet on average. Besides, we use Specweb2006 to evaluate its web service ability. TCNet can support 20% more clients simultaneously than that of Ethernet and response requests 19% faster. The results demonstrate that TCNet has great potential to accelerate cross-node virtual machine communication for data center with hybrid interconnect.