Decoupled: Low-Effort Noise-Free Execution on Commodity Systems

Today's high-performance computing (HPC) landscape is dominated by clusters built from commodity hardware. The nodes of these systems are essentially x86-based servers that run an operating system (OS) derived from an enterprise Linux distribution. In contrast, previous generations of supercomputers ran OSes that were designed specifically for the needs of HPC applications. The migration from these special-purpose OSes to off-the-shelf system software brought many advantages for both vendors and users, most importantly reduced costs and a larger feature set. However, it also left behind an important property: jitter-free execution of parallel programs. This jitter, often called OS noise, causes slowdowns for many important applications and is expected to become a major obstacle to exascale computing. Therefore, several OS research projects aim at building light-weight kernels that provide HPC applications with a noise-free execution environment. Linux runs next to these new kernels and provides functionality that they (intentionally) do not implement. However, building these new kernels and all the required support infrastructure requires considerable development and maintenance effort. We argue that a noise-free HPC OS can be built upon existing components with much less effort. In this paper, we describe a node OS that combines an off-the-shelf microkernel with a virtualized Linux kernel that provides rich functionality, including device drivers. We extended these two building blocks with a simple mechanism to decouple program execution from noisy Linux. We evaluate our prototype on a recently installed InfiniBand cluster.