The consequences of fixed time performance measurement

Abstract

In measuring the performance of parallel computers, the usual method is to choose a problem and test the execution time as the processor count is varied. This model underlies definitions of 'speedup,' 'efficiency,' and arguments against parallel processing such as Ware's (1972) formulation of Amdahl's law (1967). Fixed time models use problem size as the figure of merit. Analysis and experiments based on fixed time instead of fixed size have yielded surprising consequences: the fixed time method does not reward slower processors with higher speedup; it predicts a new limit to speedup, which is more optimistic than Amdahl's; it shows an efficiency which is independent of processor speed and ensemble size; it sometimes gives non-spurious superlinear speedup; it provides a practical means (the SLALOM benchmark) of comparing computers of widely varying speeds without distortion.<>