Toward an Adaptive Fair GPU Sharing Scheme in Container-Based Clusters

Abstract

Virtualization is an innovative technology that accelerates software development by providing portability and maintainability of applications. However, it often leads underperformance especially caused by overheads from managing virtual machines. To address the limitation of virtual machines, container technology has emerged to deploy and operate distributed applications without launching entire virtual machines. Unfortunately, resources contention issues in container-based clusters, bringing substantial performance loss are still challenging. This paper proposes an adaptive fair-share method to share effectively in container-based virtualization environment. In particular, we focus on enabling GPU sharing between multiple concurrent containers without lack of GPU memory. We demonstrate that our approach contributes to overall performance improvement as well as higher resource utilization compared to default and static fair-share methods with homogeneous and heterogeneous workloads. Compared to two other conditions, their results show that the proposed method reduces by 16.37%, 15.61% in average execution time and boosts approximately by 52.46%, 10.3% in average GPU memory utilization, respectively.