Use of PVFS for efficient execution of jobs with pipeline-shared I/O

Abstract

This paper is concerned with efficient execution of applications that are composed of chain of sequential data processes, which exchange data through a file system. We focus on pipeline-shared I/O behavior within a single pipeline of processes running on a cluster. We examine several scheduling strategies and experimentally evaluate them for efficient use the parallel virtual file system (PVFS) as a common storage pool.