Search-based Job Scheduling for Parallel Computer Workloads

Abstract

To balance performance goals and allow administrators to declaratively specify high-level performance goals, we apply complete search algorithms to design on-line job scheduling policies for workloads that run on parallel computer systems. We formulate a hierarchical two-level objective that contains two goals commonly placed on parallel computer systems: (1) minimizing the total excessive wait; (2) minimizing the average slowdown. Ten monthly workloads that ran on a Linux cluster (IA-64) from NCSA are used in our simulation of policies. A wide range of measures are used for performance evaluation, including the average slowdown, average wait, maximum wait, and new measures based on excessive wait. For the workloads studied, our results show that the best search-based scheduling policy (i.e., DDS/lxf/dynB) reported here simultaneously beats both FCFS-backfill and LXF-backfill, each roughly providing a lower bound on maximum wait and the average slowdown, respectively, among backfill policies