Performance Experiences From Running An E-health Inference Process As FaaS Across Diverse Clusters

Abstract

In this paper we report our experiences from the migration of an AI model inference process, used in the context of an E-health platform to the Function as a Service model. To that direction, a performance analysis is applied, across three available Cloud or Edge FaaS clusters based on the open source Apache Openwhisk FaaS platform. The aim is to highlight differences in performance based on the characteristics of each cluster, the request rates and the parameters of Openwhisk. The conclusions can be applied for understanding the expected behavior of the inference function in each of these clusters as well as the effect of the Openwhisk execution model. Key observations and findings are reported on aspects such as function execution duration, function sizing, wait time in the system, network latency and concurrent container overheads for different load rates. These can be used to detect in a black box manner capabilities of unknown clusters, guide or fine-tune performance models as well as private cloud FaaS deployment setup.