Effect of an Incentive Implementation for Specifying Accurate Walltime in Job Scheduling

Abstract

Backfill is a widely adopted scheduling technique in shared large scale systems. Accurate estimates of walltime of jobs benefit both users and operators of such systems because backfill uses the estimated walltime for scheduling decisions. However, reports on the accuracy analyses have shown that the accuracy is very low, which causes low utilization and long wait time. To overcome this situation, we propose to implement incentives for users to request accurate walltime in scheduling policy. We introduce a measure, named WRSA (Walltime Request Specification Accuracy), which represents the accuracy of requested walltime of each user and propose WRSA-aware backfill where jobs submitted by users with high WRSA are prioritized in scheduling. Through simulation using synthetic and real workloads, we confirm that utilization is improved up to 30% and the incentive for specifying accurate walltime is also improved against existing methods.