AdaGap: An adaptive gap-aware resource allocation strategy for GPU sharing in heterogeneous clusters

Heterogeneous GPU clusters are crucial for high-performance computing and deep learning tasks, offering a flexible and cost-effective platform. GPU sharing allows multiple containers to concurrently access the same physical GPU, improving overall GPU usage. However, underutilization of GPU resources remains a significant challenge, primarily due to inefficient resource allocation and fragmentation within GPU sharing environments. Existing GPU sharing solutions often overlook the importance of effective resource allocation strategies, leading to resource gaps. In this paper, we propose AdaGap, an adaptive gap-aware, Deep Q-Network-based resource allocation strategy designed to optimize GPU usage by minimizing underutilized gaps in heterogeneous clusters. We develop a dynamic, gap-aware resource allocation mechanism that adapts to changing task requirements and diverse GPU and CPU resources, formulating the allocation problem as a Markov Decision Process. We conduct experiments using real-world data from Alibaba cloud, and the results demonstrate AdaGap’s robust adaptability across various heterogeneous scenarios. The method improves allocation strategies by minimizing resource gaps and reducing job completion times compared to baseline methods.