Benchmarking Serverless Computing: Performance and Usability

With the increasing adoption of serverless computing, there is a need for a benchmark. The aim of this paper is to present such a benchmark based on performance and usability testing to better understand serverless services as well as help practitioners to select between two major clouds, namely, Amazon and Azure. Jmeter tool and system usability scale are used to conduct performance and usability testing, respectively. In addition, a replication package is provided to increase the validity and reliability of the results. The main findings revealed that the serverless platforms are different in their architecture. Even though both of them support the same serverless concept, they differ considerably in structure, development, and creations of services. Overall, both the cloud vendors under study provide the same core capabilities one would expect but, there are some differences too. In particular, usability could be improved to extend the market and capture more customers.