An Approach to Compiling Single-point Iterative Programs for Distributed Memory Computers

Abstract

This paper proposes a scheme for compiling an important class of iterative algorithms into efficient code for distributed memory computers. The programmer provides a description of the problem in Spot: a data parallel SIMD language that uses iterations as the unit of synchronization and is based on grids of data points. The data parallel description is in terms of a single point of the data space, with implicit communication semantics, and a set of numerical boundary conditions. The compiler eliminates the need for multi-tasking by ‘(expanding” the single-point code into multiplepoint code that executes over rectangular regions of points. Using rectangle intersection and difference operations on these regions allows the compiler to automatically insert the required communication calls and to hide communication latency by overlapping comput at ion and communication. The multiple-point code may be specialized, at compile-time, to the size and shape of different allocations, or it may use table-driven for-loops to adapt, at run-time, to the shape and size of the allocations. We show how to generalize this strategy to produce code for the near-rectangular shaped allocations required for balanced partitionings of rectangular arrays.