Run-time statistical estimation of task execution times for heterogeneous distributed computing

Abstract

An efficient run time, statistical scheme for estimating the execution time of a task is presented, in order to facilitate run time matching and scheduling in a distributed heterogeneous computing environment. This scheme is based upon a nonparametric regression technique, where the execution time estimate for a task is computed from past observations. Furthermore, this technique is able to compensate for different parameters upon which the execution time depends, and does not require any knowledge of the architecture of the target machine. It is also able to make accurate predictions when erroneous data is present in the set of observations, and has been experimentally shown to produce estimates with very low error even with few past values from which to calculate a new estimate.