ACOPT: Adaptive continuity-aware address translation for performance optimization of MCM-GPU architectures

Multi-Chiplet-Module (MCM) designs become a rising technique to boost the performance of GPUs in the post-Moore era. In MCM-GPUs, multiple chiplets are integrated within a single physical package via high-bandwidth and low-latency in-package interconnections to provide more computational and storage resources, compared with traditional monolithic GPUs. Furthermore, MCM-GPUs can adopt Unified Virtual Memory (UVM) for data management, enabling them to be programmed as logically single GPUs. In this paper, we investigate the address translation process and analyze its bottlenecks in UVM MCM-GPUs. Our profiling of a set of GPU workloads reveals that L2 TLB miss requests incur significant PCIe transfer delays, page table walkers waiting time, and page table walk latency, which are not easily hidden. In addition, the address translation process in UVM-enabled MCM-GPUs remains inefficient as they neglect spatial continuity in memory access patterns and rigid translation granularity. Based on these observations, we propose ACOPT, a continuity-aware address translation framework for MCM-GPUs. ACOPT employs a hardware-based design to adaptively capture continuous multi-granularity pages and store them into one page table entry (PTE), which releases multiple pending requests waiting for page table walkers in advance to reduce the number of page table walks. In this way, multiple pages are fetched into one L2 TLB entry to effectively extend the L2 TLB reach when L2 TLB miss happens. Our experimental results reveal that ACOPT is able to achieve 1.54$\times$ speedup, 78% reduction in number of page table walks, and 70% reduction of L1 TLB misses latency on average across a set of applications.