Supercomputing operating systems: a naive view from over the fence

Abstract

To exaggerate unfairly, from the perspective of mainstream OS research, the supercomputing community has a very different idea of the role (and appropriate design) of an OS. HPC people regard the OS as an annoying source of noise, whereas the former crowd see it as a thing of wondrous beauty and elegance, a sine qua non of usable everyday computing. This situation has existed without serious conflict erupting for years: OS researchers worried about PCs with one core (or, at most, a handful of cores) running a general-purpose OS and supporting a dynamic, bursty, diverse mix of hundreds of interactive, long-running, soft-realtime and/or background processes. Supercomputing people wanted one, highly parallel, program to finish as soon as possible so they could get on to the next one. With multicore, this all changed: highly parallel tasks will be the norm for future general-purpose computing. In 2007, my colleagues and I eagerly embarked on a new research OS for multicore computing, and looked forward to applying long-ignored (in our field) results from the HPC realm to our system. It didn't quite work out that way. In this talk I will look at what we found to be common to the two fields, and what we didn't, and speculate on where this might be going. I think there is a useful conversation to be had, and I'd like to help revive it.