Scheduling Scientific Workloads in Private Cloud: Problems and Approaches

Public cloud providers are using the "pay-per-use" model when providing their resources to customers. Among other advantages, it allows the provider to react to changing demands, e.g., by modifying prices or by extending its physical capacities using the profit obtained. In this paper we deal with a completely different model. We describe a private scientific cloud where resources are provided to researchers for free. As we demonstrate, the "absence of money" means that the system must employ other mechanisms to guarantee reasonable performance and utilization. Especially, the problem of guaranteeing user-to-user fairness represents a major issue. Moreover, since there is no financial burden related to the use of cloud infrastructure, many resources can be wasted by long running idle virtual machines (VM) that their users no longer need. This leads to underutilization and resource fragmentation. This paper discusses these problems using real-life data from the CERIT Scientific Cloud and proposes several techniques to guarantee fair and efficient use of system resources. Furthermore, we present a prototype of a new experimental OpenNebula-compatible VM scheduler which was designed as a replacement for the default scheduler provided in OpenNebula distribution. Unlike the default scheduler, our new scheduler provides complex fair-sharing mechanisms as well as modular and easy-to-extend architecture to enable further development of advanced VM scheduling policies.