Dynamic Multi-Resource Monitoring for Predictive Job Scheduling with ScoPro

Abstract

Modern job schedulers move towards applying dynamic approaches like time sharing or adaptive resource allocation to accommodate grid jobs or to better utilize local resources. Also, the resources may be heterogeneous and a proper distribution of the application's workload be hard to estimate. Our ScoPro monitoring tool permits to obtain and to store resource-related behavior information for parallel applications. This information is used to create an application signature for predictive use in future runs and to dynamically check competition under time-shared execution and imbalances of workload on heterogeneous resources. ScoPro is applicable to production runs on standard clusters. As main innovative contributions ScoPro can be triggered by job-scheduling events, can monitor several coscheduled jobs concurrently for accurate prediction of slowdowns, and performs realtime short-period measurements with low intrusion during the monitoring, while avoiding any intrusion overhead for the non-monitored part of the job execution