On-demand provisioning of long-tail services in distributed clouds

Abstract

We see a trend to design services as a suite of small service components instead of the typical monolithic nature of classic web services, which led to an increasing amount of long-tail services on the Internet. Deploying instances everywhere to achieve a fast response time results in high costs, especially when these services are used infrequently and remain idle most of the time. One way to avoid needless over-provisioning is to deploy instances on-demand but this requires every component to be available upon request arrival. We propose a placement algorithm to maximize the amount of clients we can serve on-demand using the Docker layered filesystem. Docker facilitates automated deployment of services in lightweight software containers, allowing almost instantaneous deployment. Our algorithm finds the optimal storage location for layers so we can retrieve all service layers, deploy a service instance and provide a first response to a request within the desired time. We solve this problem using integer linear programming (ILP) and present techniques to improve the scalability of ILP while minimizing the performance loss. Results show that our approximation performs better with large scale problems than the classic ILP case.