hClock: hierarchical QoS for packet scheduling in a hypervisor

Abstract

Higher consolidation ratios in virtualized datacenters, SSD based storage arrays and converged IO fabric are mandating the need for more flexible and powerful models for physical network adapter bandwidth allocation. Existing solutions such as fair-queuing mechanisms, traffic-shapers and hierarchical allocation techniques are insufficient in terms of dealing with various use cases for a cloud service provider. In this paper, we present the design and implementation of a hierarchical bandwidth allocation algorithm called hClock. hClock supports three controls in a hierarchical manner: (1) minimum bandwidth guarantee, (2) rate limit and (3) shares (a.k.a weight). Our prototype implementation in VMware ESX server hypervisor, built with several optimizations to minimize CPU overhead and increase parallelism, shows that hClock is able to enforce hierarchical controls for a variety of workloads with diverse traffic patterns at scale.