Optimal routing for distributed computing systems with data replication

Abstract

In a Distributed Computing System (DCS), data files are often replicated several copies and stored on different computers to improve reliability and availability. The particular program and the data files required for its execution may thus be placed on different nodes. Successfully executing the program, however, often requires the access of data files from remote machines. Thus, the program execution time is affected by the constraint of communication capacity. Optimal routing thus becomes an important research goal. If data are not replicated, the problem of optimal routing is essentially a multi-commodity flow problem and can be solved by linear programming technique. When the data are replicated, the program can read the data from any replicas. Reads from replicas on different nodes will results in different transmission time. An exhaustive approach could of course find the optimal solution. However, this approach has a high computation price. In this paper, we propose a method that can effectively decide the optimal traffic routing. Examples are provided to demonstrate the applicability of the proposed algorithm. Results obtained from the proposed algorithm are encouraging.