Tight bounds on one- and two-pass MapReduce algorithms for matrix multiplication

We study one- and two-pass mapReduce algorithms for multiplying two matrices. First, consider one-pass algorithms. In the literature, there is a tight bound for the tradeoff between communication cost and parallelism. It measures communication cost using the replication rate r, and measures parallelism by reducer size q. It gives a tight bound on qr for multiplying dense square matrices. We extend it in two different ways: First, to sparse rectangular matrices; second, to a different measure of parallelism, namely, reducer workload w. We present tight bounds on qr and wr2, for multiplying sparse rectangular matrices. We also show that the lower bound on qr follows from the lower bound on wr2; so, the lower bound on wr2 is stronger. Next, consider two-pass algorithms. It has been shown that, for a given reducer size, the two-pass algorithm has less communication cost than the one-pass algorithm. We present tight bounds on qfrfrs and wfr2frs, for multiplying dense rectangular matrices; the subscripts f and s correspond to the first and second pass, respectively. Also, using our bound on qfrfrs, we present a tight bound on the total communication cost as a function of qf. Our lower bounds hold for the class of two-pass algorithms that perform all the real number multiplications in the first pass.