A Scalable Multi-Replication Framework for Data Grid

Abstract

Existing replica services on the Grid we know to date assumes point-to-point communication and file transfer protocol. As such, when hundreds to thousands of hosts on the Grid access a single dataset simultaneously, bottlenecks in networks and/or the data servers will hinder performance significantly. Instead, our replication framework couples efficient, multicast techniques with a replica catalog that automatically detects simultaneous access to the replica by multiple nodes. As a prototype, we have designed and built a portable, XML-based replica location service accounting for such parallel transfer requests, and coupled it with a O(1) bulk file transfer system Dolly+[6]. The benchmarks show that the system is scalable and effective in reducing replication costs significantly in cluster-based replication scenarios.