Dynamic Load Balancing in Kubernetes Environments With Kubernetes Scheduling Extension (KSE)

Abstract

Kubernetes is a flexible and reliable container orchestrator that has been employed to maintain massive cloud infrastructures worldwide. The task of allocating “Pods” (deployable units of computing that have one or more containers) to cluster nodes in Kubernetes is done by the Kube-Scheduler module, which determines which nodes are valid placements for each Pod according to constraints and available resources. Since it only acts at Pod creation, it does not take any action when the load of the system becomes uneven. In imbalanced scenarios, overloaded nodes can compromise the performance and availability of services hosted by them, whereas underloaded nodes may be a waste of financial resources, especially when using public clouds. In this paper, we propose an extension to the Kubernetes scheduler, called Kubernetes Scheduling Extension (KSE), which allows users to implement dynamic load-balancing algorithms that can migrate Pods between nodes at runtime. We also provide the implementation of two well-known load-balancing algorithms (KSE-GreedyLB and KSE-RefineLB) in KSE, which can balance the load of system nodes using CPU and memory consumption metrics. We carried out several experiments to assess the effectiveness of KSE-GreedyLB and KSE-RefineLB and compared their results with Kube-Scheduler. Overall, we evaluated 32 different scenarios using synthetic and realistic applications. Our results showed that KSE-RefineLB achieves better results than Kube-Scheduler when the workload is highly imbalanced while keeping similar performance when the load imbalance is low.