Understanding Parallel I/O Performance Trends Under Various HPC Configurations

In high-performance computing (HPC) environments, an appropriate amount of hardware resources must be used for the best parallel I/O performance. For this reason, HPC users are provided with tunable parameters to change the HPC configurations, which control the amounts of resources. However, some users are not well aware of a relationship between the parallel I/O performance and the HPC configuration, and they thus fail to utilize these parameters. Even if users who know the relationship, they have to run an application under every parameter combination to find the setting for the best performance, because each application shows different performance trends under different configurations. The paper shows the result of analyzing the I/O performance trends for HPC users to find the best configurations with minimal efforts. We divide the parallel I/O characteristic into independent and collective I/Os and measure the I/O throughput under various configurations by using synthetic workload, IOR benchmark. Through the analysis, we have figured out that the parallel I/O performance is determined by the trade-off between the gain from the parallelism of increased OSTs and the loss from the contention for shared resources. Also, this performance trend differs depending on the I/O characteristic. Our evaluation shows that HPC applications also have similar performance trends as our analysis.