CEMA: Cost Effective Multi-Layered Autoscaling for Microservice based Applications

Abstract

Microservices architecture offers flexibility, scalability, and modularity by dividing applications into small and independent services. However, traditional autoscaling methods often focus on the autoscaling of the container layer alone, leading to inefficiencies such as over-provisioning and under-provisioning of virtual machines (VMs). These inefficiencies can increase operational costs and energy consumption. To address these challenges, this paper presents a novel, cost-effective Multi-Layered Autoscaling (CEMA) strategy that includes service migration to optimize resource allocation across container and VM layers. CEMA leverages predictive autoscaling techniques to dynamically adjust the number of containers and VMs based on real-time workload demands. The strategy includes a service migration mechanism that moves containers from underutilized VMs to those with available capacity, enabling the shutdown of idle VMs and reducing energy consumption. Through extensive experimentation using real-world workloads, including the WorldCup, Wikipedia, Calgary, ClarkNet, and NASA, CEMA demonstrates significant improvements over existing autoscaling methods. Results show CEMA gives 11.7% more processed requests with 19% fewer SLO violations than the baseline methods. Moreover, CEMA reduces the 1.6$\times$ infrastructure cost as compared to baseline methods. This paper highlights CEMA’s potential to enhance the efficiency and sustainability of microservices-based applications in cloud environments.