Fair Epoch Scheduling of Linear Workflows with High Variability in Task Service Demand

Abstract

With the evolution of the distributed computing model, the service times of the applications processed on such platforms present increased variability. Moreover, the workload in such environments has usually the structure of linear workflows (LWs). Jobs of this type consist of a set of tasks which need to be processed one after the other. LWs often result in uneven processor queues, as they attempt to leverage data locality. Consequently, it is very important to employ appropriate scheduling strategies in order to achieve good performance, as well as fairness in the workload execution. In this paper we aim to examine efficient scheduling techniques of LWs. We consider that the variability in the task service demands is high. Three scheduling schemes are employed in a distributed system. Their performance is evaluated and compared using simulation. The simulation results demonstrate the different potential of the scheduling approaches which we use. Also, they reveal that the efficacy of the scheduling schemes is influenced by, the coefficient of variation of the probability distribution of service times of the tasks of the linear workflow jobs, the load level, and whether the primary scheduling objective is fairness.