A CPU utilization limit for massively parallel MIMD computers

Massively parallel computer systems based on off-the-shelf CPU chip-sets have become commercially available. The authors demonstrate a theoretical limit on the silicon (or other circuitry media) utilization of such architectures as the number of processors is scaled up. In addition, case studies of the Thinking Machines Corporation CM-5 and of the Intel Touchstone are presented in order to quantify the maximum utilization on existing machines. Based on this utilization limit, the authors examine whether computer architects' current reliance on the MIMD (multiple-instruction multiple-data) model will be practical in next-generation machines. In order to facilitate the analysis, they decouple the control parallel and data parallel models of computation from MIMD and SIMD (single-instruction multiple-data) target platforms, respectively. Utilization of control parallel paradigms executing on SIMD platforms is introduced for comparison. The authors also consider the relationship of communication overhead to machine size scaling in the presence of the need for virtual processing nodes.<>