Beyond Load Balancing: Package-Aware Scheduling for Serverless Platforms

Abstract

Fast deployment and execution of cloud functions in Function-as-a-Service (FaaS) platforms is critical, for example, for microservices architectures. However, functions that require large packages or libraries are bloated and start slowly. An optimization is to cache packages at the worker nodes instead of bundling them with the functions. However, existing FaaS schedulers are vanilla load balancers, agnostic of packages cached in response to prior function executions, and cannot properly reap the benefits of package caching. We study the case of package-aware scheduling and propose PASch, a novel scheduling algorithm that seeks package affinity during scheduling so that worker nodes can re-use execution environments with preloaded packages. PASch leverages consistent hashing and the power of 2 choices, while actively avoiding worker overload. We implement PASch in a new scheduler for the OpenLambda framework and evaluate it using simulations and real experiments. When using PASch instead of a least loaded balancer, tasks perceive an average speedup of 1.29x, and 80th percentile latency that is 23x faster. Furthermore, for the workload studied in this paper, PASch outperforms consistent hashing with bounded loads—a state-of-the-art load balancing algorithm—yielding a 1.3x average speedup, and a speedup of 1.5x at the 80th percentile.