Cross-Platform Performance Evaluation of Stateful Serverless Workflows

Abstract

Serverless computing, with its inherent event-driven design along with instantaneous scalability due to cloud-provider managed infrastructure, is starting to become a de-facto model for deploying latency critical user-interactive services. However, as much as they are suitable for event-driven services, their stateless nature is a major impediment for deploying long-running stateful applications. While commercial cloud providers offer a variety of solutions that club serverless functions along with intermediate storage to maintain application state, they are still far from optimized for deploying stateful applications at scale. More specifically, factors such as storage latency and scalability, network bandwidth, and deployment costs play a crucial role in determining whether current serverless applications are suitable for stateful workloads. In this paper, we evaluate the two widely-used stateful server-less offerings, Azure Durable functions and AWS Step functions, to quantify their effectiveness for implementing complex stateful workflows. We conduct a detailed measurement-driven characterization study with two real-world use cases, machine learning pipelines (inference and training) and video analytics, in order to characterize the different performance latency and cost tradeoffs. We observe from our experiments that AWS is suitable for workloads with higher degree of parallelism, while Azure durable entities offer a simplified framework that enables quicker application development. Overall, AWS is 89% more expensive than Azure for machine learning training application while Azure is 2× faster than AWS for the machine learning inference application. Our results indicate that Azure durable is extremely inefficient in implementing parallel processing. Furthermore, we summarize the key findings from our characterization, which we believe to be insightful for any cloud tenant who has the problem of choosing an appropriate cloud vendor and offering, when deploving stateful workloads on serverless platforms,