PaLLOC: Pairwise-based low-latency online coordinated resource manager of last-level cache and memory bandwidth on multicore systems

Abstract

Modern advanced multicore CPUs integrate large last-level caches (LLC) and provide high memory bandwidth, which are generally shared among cores. In many scenarios, isolated resources are required among co-running applications with dynamic changes. This drives the need for online partitioning of these shared hardware resources to accommodate applications’ different and varying resource demands. However, dynamically managing LLC and memory bandwidth without prior knowledge faces numerous searches for resource configurations to gather sufficient information and find the partition solution, which may cause long management latency and limit system performance. To address this problem, we first identify several workload-independent observations and insights through a comprehensive exploration of the configuration space across various benchmarks, which can help reduce the need for configuration searches greatly. Guided by these findings, we propose a method that integrates two-step allocation with pairwise search techniques to maximize system instructions per cycle (IPC) throughput. Building on this method, we design and implement PaLLOC, a novel low-latency online coordinated resource manager of LLC and memory bandwidth on multicore systems. Comprehensive evaluations on an Intel commodity server demonstrate that PaLLOC consistently exhibits significant performance advantage across various system workloads with diverse resource requirements, achieving 1.14x-1.47x speedup in system IPC throughput over the state-of-the-art online partitioning method, with a management latency of approximately 300ms under a monitoring period of 10 ms.