MARS: a metascheduler for distributed resources in campus grids

Abstract

Computational grids are increasingly being deployed in campus environments to provide unified access to distributed and heterogeneous resources such as clusters, storage arrays, networks, and scientific instruments. While the existing grid computing frameworks and protocols provide a robust set of mechanisms for user authentication, security, workflow and resource management; efficient scheduling of tasks on distributed and heterogeneous resources, termed as metascheduling, is an active area of research. In this paper, we describe MARS, an open-source metascheduling framework that can be integrated into existing campus infrastructure to provide robust task scheduling and resource management capabilities. MARS uses a forecasting algorithm to predict resource-level scheduling parameters such as queue lengths, turn-around times, and resource utilization. These predicted values are then used to schedule tasks based on their priority levels. It allows preemption of lower-priority running tasks in favor of on-demand tasks. We have implemented heuristic and evolutionary scheduling algorithms in the present framework and evaluated it in a production environment consisting of several large Linux clusters. Our simulation results using actual workload traces from these clusters demonstrate the effectiveness of the current metascheduling framework.