Adaptive Site Selection and Task Scheduling for Grid Workflow

Abstract

The availability and performance of Grid sites change dynamically, which makes it be a great challenge to design a good site selection and task scheduling mechanism for Grid workflows. A multi-stage Grid site evaluation and selection mechanism, involving site testing, time series based site performance prediction and adaptive selection, was proposed in this paper. In order to enhance reliability of workflow execution and potentially reduce execution time, an incremental task replication policy based on empirical CDF of queuing time of sites was introduced. The experiments done in Open Science Grid (OSG) proved that adaptive site selection and task scheduling proposed in this paper could control JRR and JSL in a reasonable range in a realistic Grid environment, and workflow execution performance could be greatly improved by initial site evaluation and task replication policies.