Assessment and enhancement of meta-schedulers for multi-site job sharing

Abstract

Meta-schedulers such as the LSF MultiCluster scheduler and the Moab meta-scheduler (Silver) are being deployed on clusters interconnected over the grid. A primary goal of such meta-schedulers is to share jobs amongst the individual local sites, balancing the load and improving utilization and turnaround time. This work focuses on current methodologies used in implemented meta-schedulers, such as LSF MultiCluster and Silver. The benefits and disadvantages of both centralized and delegated modes are evaluated. Focusing on the negative impact of meta-schedulers to jobs originating from lightly loaded sites, enhancements are proposed and evaluated via trace-driven simulation. It is shown that it is feasible to reduce the detrimental impact on lightly loaded sites while maintaining excellent overall performance.