Challenges and Opportunities for Unikernels in Machine Learning Inference

Machine Learning has become a value creator for many new and old businesses. However, efficient realworld machine learning deployments are still a challenge. Traditional Machine Learning deployments suffer from efficient resource utilization and achieving predictable latency. They cannot be treated in the same manner as other application server deployments. Unikernels are a method to specialize application deployment and performance to suit the needs of the application. Traditionally, building or porting applications to unikernels have been challenging. However, recent work has been into simplifying the development of unikernels. Real-world Unikernels as of now are only for specializing applications that run on the CPU. We survey machine learning practitioners and find out that the majority of machine learning practitioners are using the CPU for machine learning deployments, thus, creating an opportunity for unikernels to optimize the performance of these applications. We compare the architecture of two unikernels: nanos and Unikraft. We benchmarked scikit-learn, a popular machine library, inside a unikernel and found that it only offered a 1% advantage over a traditional deployment. However, our testing could not include more innovative systems like Unikraft due to their immaturity and inability to run machine learning libraries. We include a dependency analysis of three popular machine learning libraries Tensorflow Lite, PyTorch and ONNX, to help pave the way for building machine learning applications as Unikraft unikernels.