Network design and performance for a massively parallel SIMD system

Abstract

It is shown that a nearest neighbor communication network can be complimented with a log-diameter multistage network to handle different communications patterns. This is especially useful when the pattern of data movement is not uniform. The designed network is evaluated for two cases: a dense case with many processing elements communicating and a sparse case. For 32-b data, the algorithm for computing partial sums of an array improves by 2.7 times with the multistage interconnection network. In a sparse random case, the number of cycles taken to communicate 32 b is 4000 (with 10% of the nodes communicating). Thus, it is concluded that a network like a multistage omega network is very useful for SIMD (single-instruction multiple-data) massively parallel machines. This is especially true if the machine is to be used for applications where long distance and nonuniform routing patterns are needed.<>