Hierarchical Page Eviction Policy for Unified Memory in GPUs

Abstract

The introduction of unified memory in discrete GPUs not only improves programmability but also enables oversubscription. However, it introduces high overhead when page faults occur. Therefore, when GPU memory is full, how to select eviction candidates becomes an important issue. The widely used policy LRU performs poorly for workloads with thrashing access patterns, and the advanced cache replacement policy RRIP incurs thrashing when directly applied to GPU memory. In this paper, we propose hierarchical page eviction policy for GPU memory, which relies on a software-managed page set chain to select eviction candidates. Results show that for 15 selected applications, our policy achieves an average speedup of 1.44 and 1.2 over LRU when the oversubscription rate is 75% and 50 %, respectively.