Life with Ed: a case study of a linux BIOS/BProc cluster

Abstract

In this paper, we describe experiences with our 127-node/161-processor Alpha cluster estbed, Ed. Ed is unique for two distinct reasons. First, we have replaced the standard BIOS on the cluster nodes with the Linux BIOS which loads Linux directly from non-volatile memory (Flash RAM). Second, the operating system provides a single-system image of the entire cluster, much like a traditional supercomputer. We will discuss the advantages of such a cluster, including time to boot (101 seconds for 100 nodes), upgrade (same as time to boot), and start processes (2.4 seconds for 15,000 processes). Additionally, we have discovered that certain predictions about the nature ofter a scale clusters, such as the need for hierrchical structure, are false. Finally, we argue that to achieve true scalability, terascale clusters must be built in the way of Ed.