Niyama: Node scheduling for cloud workloads with resource isolation

Abstract

Cloud providers place tasks from multiple applications on the same resource pool to improve the resource utilization of the infrastructure. The consequent resource contention has an undesirable effect on latency‐sensitive tasks. In this article, we present Niyama—a resource isolation approach that uses a modified version of deadline scheduling to protect latency‐sensitive tasks from CPU bandwidth contention. Conventionally, deadline scheduling has been used to schedule real‐time tasks with well‐defined deadlines. Therefore, it cannot be used directly when the deadlines are unspecified. In Niyama, we estimate deadlines in intervals and secure bandwidth required for the interval, thereby ensuring optimal job response times. We compare our approach with cgroups: Linux's default resource isolation mechanism used in containers today. Our experiments show that Niyama reduces the average delay in tasks by 3 ×$$ \times $$ –20 ×$$ \times $$ when compared to cgroups. Since Linux's deadline scheduling policy is work‐conserving in nature, there is a small drop in the server‐level CPU utilization when Niyama is used naively. We demonstrate how the use of core reservation and oversubscription in the inter‐node scheduler can be used to offset this drop; our experiments show a 1.3 ×$$ \times $$ –2.24 ×$$ \times $$ decrease in delay in job response time over cgroups while achieving high CPU utilization.