Initial Steps toward Making GPU a First-Class Computing Resource: Sharing and Resource Management

Abstract

GPUs have evolved from traditional graphics accelerators into core compute engines for a broad class of general-purpose applications. However, current commercial offerings fall short of the great potential of GPUs largely because they cannot be managed as easily as the CPU. The enormous amount of hardware resources are often greatly underutilized. We developed new architecture features to enable fine-grained sharing of GPUs, termed Simultaneous Multi-kernel (SMK), in a similar way the CPU achieves sharing via simultaneous multithreading (SMT). With SMK, different applications can co-exist in every streaming multiprocessor of a GPU, in a fully controlled way. High resource utilization can be achieved by exploiting heterogeneity of different application behaviors. Resource apportion among sharers are developed for fairness, throughput, and quality-of-services. We also envision that SMK can enable better manageability of GPUs and new features such as more efficient synchronization mechanisms within an application.