Well-Balanced Allocation Strategy for Multi-Cluster Computing Environments

Abstract

With the rapid proliferation of hardware devices, cluster computing has encountered the problem of heterogeneous resources. We have constructed a multi-cluster system that works with and without storage devices, and evaluated the system performance. We dispatch jobs with the strategy we propose to make the best use of system resources. In this paper we introduce this new scheduling system based on multi-clusters in diskless environments. This work we present is the Well-Balanced Allocation Strategy (WBAS) in which the scheduler dispatches jobs to appropriate resources across multi-clusters. The strategy focuses on dispatching jobs to nodes with similar performance capacities, thus equalizing execution times among all the nodes the jobs require. The WBAS was implemented on a system to evaluate the performance of the scheduling strategy. We use Ganglia to monitor resource statuses, and Condor as the queue system.