A parallel out-of-core computing system using PVFS for Linux clusters

Abstract

Cluster systems become a new and popular approach to parallel computing. More and more scientists and engineers use clusters to solve problems with large sets of data for its high processing power, low price and good scalability. Since the traditional out-of-core programs are difficult to write and the virtual memory system does not perform well, we develop a parallel out-of-core computing system using PVFS named POCCS. POCCS provides convenient interface to write out-of-core codes and the global view of the out-of-core data. The software architecture, data storage model and system implementation are described in this paper. The experimental results show that POCCS extends the problem sizes that can be solved and the performance of POCCS is better than the virtual memory system while the data set is large.