Smart job scheduling with backup system in grid environment

Abstract

This paper investigates the problem of job scheduling in grid environments when dependencies between the submitted jobs exist. If a job is failed, all jobs depending on it will need to be restarted. In order to prevent that, a Dependency Resolution model with a backup system (DR-Backup) is designed. DR-Backup uses Back Propagation Neural Network (BPNN) to predict the weight of the jobs. Also, it uses an unsupervised neural network to classify the slaves (working machines) into a set of classes. Three statistical predictors were used to validate the BPNN predictor as follow: Ordinary Least Square Regression (OLSR), MARS regression and the Treenet Logistic Binary predictor. Results show that the OLSR has a higher prediction rate than the other models. DR-Backup model was compared with three methods in job scheduling: First Come First Serve (FCFS), Job Ranking Backfilling (JR-Backfilling) and SLOW-coordination. Results show that no algorithm can overcome all dynamics in the incoming jobs and any system has advantages and disadvantages depending on the jobs sample and the parameters that were taken in classifying incoming jobs.