Experiment Management and Analysis with perfbase

Abstract

Achieving the desired performance with application software, middleware or operating system components on a parallel computer like a cluster is a complex task. Typically, a high-dimensional parameter space has to be reduced to a small number of core parameters, which influence the performance most significantly, but still a large number of experiments is necessary to determine how the best performance can be achieved. Keeping track of these experiments to derive the correct conclusions is a major task. This paper presents perfbase, a set of front end tools and an SQL database as backend, which together form a system for the management and analysis of the output of experiments. In this context, an experiment is an execution of an application or library on a computer system. The output of such an experiment are one or more text files containing information on the execution of the application. This output is the input for perfbase which extracts specified information to store it in the database and make it available for management and analysis purposes in a consistent, fast and flexible manner