CASR: Optimizing cold start and resources utilization in serverless computing

Serverless computing, also known as Functions as a Service (FaaS), is an emerging cloud deployment paradigm that offers advantages such as pay-as-you-go pricing and automatic scaling. Functions often suffer from cold starts delays due to the overhead of initializing code and data dependencies before execution. Retaining containers in memory for a period after execution can reduce cold start latency. However, existing application-layer solutions overlook the effect of cold start overhead on the availability of containers, resulting in suboptimal balance between cold start latency and memory resource utilization. Furthermore, these strategies typically overlook the optimization of overall cold start overhead, which is essential for enhancing system efficiency. To address these challenges, we propose the Cache-Based Adaptive Scheduler for Serverless Runtime (CASR), an adaptive strategy for managing container runtime configurations. CASR effectively balances cold start latency and memory utilization while reducing overall cold start overhead. Specifically, we introduce a serverless cache (S-Cache) that leverages the equivalence between caching problems and container keep-alive strategies to mitigate cold starts. Additionally, we develop a deep reinforcement learning model, based on the proximal policy optimization algorithm, to enable the automatic scaling of the S-Cache queue, allowing adaptation to dynamic cloud workloads and enhancing memory resource utilization. Extensive simulations on an Azure dataset show that CASR reduces cold starts by 38.75%, improves memory resource utilization by 46.73%, and decreases cold start overhead by 48.53% compared to existing container keep-alive strategies in serverless platforms under common workloads.