Equalizer: Energy-efficient machine learning-based heterogeneous cluster load balancer

Abstract

Heterogeneous systems deliver high computing performance when effectively utilized. It is crucial to execute each application on the most suitable device while maintaining system balance. However, achieving equal distribution of the computing load is challenging due to variations in computing power and device architectures within the system. Moreover, scheduling applications at real-time further complicates this task, as prior information about the submitted applications is absent. In this context, we introduce “Equalizer,” a real-time load balancer for heterogeneous systems. “Equalizer” leverages machine learning to continuously monitor the system's state, predicting optimal devices for application execution at runtime. It assigns applications to devices that minimize system imbalance. To quantify system imbalance, we propose a novel metric that reflects the disparity in computing loads across the system's devices. This metric is calculated using predicted execution times of applications. To validate the performance of “Equalizer,” we conducted a comparative study against widely adopted approaches, namely Round Robin and Device Suitability. The experiments were performed on a heterogeneous cluster comprising a master host and three slave servers, equipped with a total of 4 central processing units (CPUs) and 4 graphics processing units (GPUs). All approaches were deployed on the cluster and evaluated using three distinct workloads categorized by their computing intensity: medium intensity, heavy intensity, and a combination of heavy and medium intensity, simulating real-world scenarios. Each workload consisted of a set of 80 OpenCL applications with varying input data sizes. The experimental results demonstrate that “Equalizer” effectively minimized the system's imbalance, reduced the idle time of devices, and eliminated overloads. Moreover, “Equalizer” exhibited significant improvements in workload execution time, resource utilization, throughput, and energy consumption. Across all tested scenarios, “Equalizer” consistently outperformed alternative approaches, showcasing its robustness, adaptability to dynamic environments, and applicability in real-world practice.