A dynamic load dispersion algorithm for load-balancing in a heterogeneous grid system

Abstract

The ever changing demands on computational resources has information systems managers looking for solutions that are more flexible. Using a ldquobigger boxrdquo that has more and faster processors and permanent storage or more random access memory (RAM) is not a viable solution as the system usage patterns vary. In order for a system to handle the peak load adequately, it will go underutilized much of the time. A grid based distributed system can solve this problem by allowing multiple independent jobs to run over a network of heterogeneous computers. Applications can be based on several parallel jobs, several sequential jobs, or a single job. Keeping the workload represented by these jobs balanced over the network of computers requires network-aware scheduling algorithms that are dynamic, transparent, scalable, and quick. We present such an algorithm that handles load-balancing of jobs submitted at any point in the grid. Our algorithm accommodates jobs with differing CPU and I/O requirements and load-balances them over varying grid loads and varying network latencies.