Runtime data declustering over SAN-connected PC cluster system

Abstract

Personal computer/workstation (PC/WS) clusters have come to be studied intensively in the field of parallel and distributed computing. From the viewpoint of applications, data intensive applications including data mining and ad-hoc query processing in databases are considered very important for massively parallel processors, in addition to the conventional scientific calculation. Thus, investigating the feasibility of such applications on a PC cluster is meaningful. A PC cluster connected with a storage area network (SAN) is built and evaluated with a data mining application. In the case of a SAN-connected cluster, each node can access all shared disks directly without using a LAN; thus, SAN-connected clusters achieve much better performance than LAN-connected clusters for disk-to-disk copy operations. However, if a lot of nodes access the same shared disk simultaneously, application performance degrades due to the I/O-bottleneck. A runtime data declustering method, in which data is declustered to several other disks dynamically during the execution of the application, is proposed to resolve this problem.