Measuring I/O Performance of Lustre and the Temporary File System for Tradespace Applications on HPC Systems

Abstract

Tradespace analysis is an important part of design and modeling simulations in the military, in which thousands of design parameters are made available for detecting potential areas for performance enhancement in expensive equipment such as combat and transport vehicles. In pursuit of larger, more accurate tradespaces, engineers port existing serial design and evaluation codes onto ERDC's supercomputers. This results in a significant number of producer-consumer, file-mediated codes running simultaneously on Lustre, the distributed file system underlying our HPC systems. In this paper, we compare the performance of Lustre to Linux tmpfs, a shared memory file system that was designed for interprocess data transfer within a single node. Our results suggest that when conducting many file I/O operations in close succession, tmpfs offers tremendous speedup over Lustre-controlled disk storage. For tradespace applications, our results lend credence to a strategy of scheduling and organizing disparate codes in such a way to leverage tmpfs. We believe our results will help guide engineers at other national labs and scientists without HPC expertise to more easily port their I/O heavy serial codes to HPC systems.