Improving Grid Scheduling of Pipelined Data Processing by Combining Heuristic Algorithms and Simulated Annealing

Abstract

To improve the performance of pipelined data processing on computational grids, the method combining simulated annealing with a set of heuristic algorithms is presented to optimize grid scheduling. Pipelined data processing is divided into multiple sub-applications, and every sub-application is supposed moldable. Thus, sub-applications should be assigned onto their appropriate grid nodes, while parallel degrees should be determined reasonably. On one grid node, sub-applications are supposed to spatially share processor resources, and a set of heuristic algorithms is presented to optimize parallel degrees for different performance parameters respectively, based on which simulated annealing is simplified for optimizing sub-application assignments. Experiments show that the throughput or latency of pipelined data processing can be efficiently improved by the optimization of grid scheduling