Reversing the supermarket: A distributed approach for handling elasticity in the cloud

A fundamental capability of cloud computing is elasticity, i.e., the ability to dynamically change the amount of allocated resources. This is typically done by adjusting the number of Virtual Machines (VMs) running a service based on the current demand for that service. For large services, centralized management is impractical and distributed methods are employed. In such settings, no single component has full information on the overall demand and service quality, thus elasticity becomes a real challenge. We address this challenge by proposing a novel elasticity scheme that enables fully distributed management of large cloud services. Our scheme is based on three main components, namely, a task assignment policy, a VM scale-up policy and a VM scale-down policy. The task assignment policy strives to “pack” VMs while maintaining SLA requirements. The VM scale-up policy is based on local activation of new VMs and the VM scale-down policy is based on self-deactivation of VMs that are idle for some duration of time. Through simulations and an implementation we establish that our scheme quickly adapts to changes in job arrival rates and minimizes the number of active VMs so as to reduce the operational costs of the service, while adhering to strict SLA requirements.