ISACPP: Interference-Aware Scheduling Approach for Deep Learning Training Workloads Based on Co-Location Performance Prediction

Abstract

Traditional exclusive cloud resource allocation for deep learning training (DLT) workloads is unsuitable for advanced GPU infrastructure, leading to resource under-utilization. Fortunately, DLT workload co-location provides a promising way to improve resource utilization. However, existing workload co-location methods fail to accurately quantify interference among DLT workloads, resulting in performance degradation. To address this problem, this article proposes an interference-aware scheduling approach for DLT workloads based on co-location performance prediction, dubbed ‘ISACPP’. ISACPP first builds an edge-fusion gated graph attention network (E-GGAT) that incorporates DL model structures, underlying GPU types, and hyper-parameter settings to predict co-location performance. Since the co-location state changes as each workload is completed, ISACPP proposes a multi-stage co-location interference quantification model derived from the predicted co-location performance to identify the GPU device with the minimum overall interference. Experimental results demonstrate that ISACPP can accurately estimate the co-location performance of DLT workloads with a maximum prediction error of 8.72%, 1.9%, and 4.4% for execution time, GPU memory consumption, and GPU utilization, respectively. Meanwhile, ISACPP can significantly shorten workload makespan by up to 34.9% compared to state-of-the-art interference-aware scheduling methods.