Stateful Depletion and Scheduling of Containers on Cloud Nodes for Efficient Resource Usage

Container scheduling is a fundamental part of today’s service and cloud-based applications. Schedulers operate at different levels depending on how much control the system developers have. On the one hand, container orchestration managers such as Google Kubernetes manage the scheduling of containers to different nodes. On the other hand, serverless managers, such as Google Autopilot, take care of the underlying infrastructure automatically, and developers do not need to manage the nodes. However, when it comes to container depletion, i.e., removing the assigned cloud resources to an idle container, current scheduling technologies have limitations. In this paper, we propose our approach to managing cloud resource usage when containers are idle efficiently. For this purpose, we deplete idle containers statefully, i.e., propose a novel manager that monitors idle containers, saves their state, and efficiently depletes them. This manager reconstructs a depleted container using the saved state when reconstruction is needed. In our approach, we suggest an Infrastructure as Code component to automate the creation of new nodes if a depleted container cannot be scheduled on the same node, e.g., because of being overloaded. We provide an analytical model for the stateful depletion of containers and their rescheduling and empirically evaluate the accuracy of our model. For this purpose, we ran an experiment on a private cloud infrastructure and Google Cloud Platform. Our model has a low error rate of 4.28% averaged over public and private clouds.