Parallel I/O: a set of intertwined systems and applications issues

Abstract

Historically, input/output (I/O) has been a bottleneck for many scientific and engineering applications. Sequential and vector supercomputers invested substantial resources into reducing that bottleneck. Now that high-performance computing is nearly synonymous with parallel computing, I/O has become more of a bottleneck and it is more difficult to eliminate it because of the complexity of data movements between hundreds or thousands of processors and dozens or hundreds of I/O devices. This problem is starting to be addressed by various research and development efforts and by computer vendors. Some early efforts and lessons learned are presented.