OPAL: An Open-Source MPI-IO Library over Cray XT

Abstract

Parallel I/O over Cray XT is supported by a vendor-supplied MPI-IO package. This package contains a proprietary ADIO implementation built on top of the sysio library. While it is reasonable to maintain a stable code base for users' convenience, it is also very important for the system developers and researchers to analyze and assess the effectiveness of parallel I/O software, and accordingly, tune and optimize the MPI-IO implementation. A proprietary parallel I/O code base relinquishes such flexibilities. On the other hand, a generic UFS-based MPI-IO implementation is typically used on many Linux-based platforms, which lacks of the capability of utilizing Lustre specific features. We have developed an open- source MPI-IO package over Lustre, referred to as OPAL (Opportunistic and Adaptive MPI-IO Library over Lustre). OPAL provides a single source-code base for MPI-IO over Lustre on both the Cray XT and Linux platforms. It provides a number of good features, including arbitrary specification of striping patterns and Lustre-stripe aligned file domain partitioning. This paper presents the performance comparisons between OPAL and Cray's proprietary implementation. Our evaluation demonstrates that OPAL achieves the performance comparable to the Cray implementation. We also exemplify the benefits of OPAL in profiling the internal time breakdown of the collective I/O protocol over Cray XT.