Transparent Optimization of Parallel File System I/O via Standard System Tool Enhancement

Abstract

Standard system tools employed by users on a daily basis do not take full advantage of parallel file system I/O bandwidth and do not understand associated idiosyncrasies such as Lustre striping. This can lead ton on-optimal utilization of both the user's time and system resources. This paper describes a set of modifications made to existing tools that increase parallelism and automatically handle striping. These modifications result insignificant performance gains in a transparent manner with maximum speedups of 27×, 15×, and 31× for parallelized cp, tar creation, and tar extraction, respectively.