Online Job Provisioning for Large Scale Science Experiments over an Optical Grid Infrastructure

Abstract

Many emerging science experiments require that the massive data generated by big instruments be accessible and analyzed by a large number of geographically dispersed users. Such large scale science experiments are enabled by an Optical Grid infrastructure which integrates Grid software with a WDM network. This paper studies the following problem in an Optical Grid environment: given an online job request, how to optimally find a host to execute the job, taking into account the need to stage missing input files stored at other places, with the goal of satisfying the job's QoS requirements, subject to dynamic computing and network resource usage status? We first formulate the optimization problem as a Mixed Integer Linear Programming (MILP). As the MILP solution quickly gets intractable when the network size grows larger, we also propose an adaptive heuristic called AOJP. Our simulation results demonstrate both the effectiveness and the efficiency of AOJP.