Computer Benchmarks: 50 Years Ago and Now

Abstract

Fifty years ago, computer benchmarks were a common practice. In a 1973 article in Computing Surveys, Eric Timmreck, a recent University of Wisconsin-Madison CS Ph.D. graduate working at Shell Research in Houston, Texas, identified three benchmark types to validate a proposed system: Artificial, Standard, and Live benchmarks [1]. He highlighted the latter type in saying, "There is substantial agreement that the only performance tool which is accurate enough for selection purposes is a well-composed live benchmark." Timmreck left it to readers to pick a methodology, including any of six other approaches he cited "to prove that a system can in fact carry the user's load."