Containerizing HPC Applications on Heterogeneous Systems for Centralized Resource Management: A Case Study

Abstract

Recently, the demand for scientific computing on HPC systems has grown in popularity. However, the runtime environment is a standpoint when there are many kinds of different applications with different requirements. Moreover, an HPC system cannot satisfy all of these requirements of environment. This becomes more and more considerable in the case of applications running on heterogeneous systems (e.g., CPU/Intel Xeon Phi based cluster). Generally, two main problems needing to be tackled in HPC systems are runtime environment and workload management. In terms of lightweight virtualization, Docker facilitates the isolation of different applications as well as runtime environments on the same host operating system. In addition, with huge advantages, batch job scheduler plays a vital role in management and operation. In this paper, we adopt an approach by combining containerization and HPC workload management to support the submission of a variety of applications. Practically, we perform the experiments on a heterogeneous cluster with CPU and Intel Xeon Phi coprocessor. The results show that there is a slightly different about the performance of jobs which are submitted by the normal way and containerized way. However, the experimental result highlights that the cost of containerizing HPC applications is negligible, and this can be applied in practice to fulfill user's requirement.