Weighted Load Balancing Method for Heterogeneous Clusters on Hybrid Clouds

Abstract

In recent years, edge device and AI services have been expected to utilize scalable cloud computing to handle large amounts of processing. In the cloud, load-balancing techniques distribute the load evenly to many nodes to achieve high throughput. At the same time, the shift to hybrid cloud computing requires additional nodes with different generations or different types of computational resources to achieve high performance in an environment with heterogeneous computational performance. Heterogeneity raises concerns that the current uniform load-balancing will result in overloaded or underloaded nodes, which degrade responsiveness. Therefore, this study proposes a weighted load-balancing method to improve responsiveness in clusters with nonuniform computational performance. The proposed method is effective in improving the average response time by about 20%, the maximum response time by about 45%, and the response time variance by about 70% compared to load-balancing with a load balancer developed by Google.