Node-based memory management for scalable NUMA architectures

Large state-of-the-art NUMA systems may offer more than two levels of node distances. The result is a hierarchical architecture with significant differences in memory access bandwidth and latency. Consequently, NUMA-aware memory management and the reduction of remote memory accesses becomes more and more the key challenge for the operating system and its applications. In this paper, we will show that traditional, centralized concepts to realize paging are not longer an adequate approach for these architectures. We present a prototype of new node-based memory management for the Linux kernel and prove its scalability and usability. The hardware architecture is reflected by managing one page mapping table per NUMA node and the kernel's page fault handler is extended to create node-local references. Based on this prototype, we suggest extensions to simplify the detection of performance issues, which will increase the usability of such architectures.