New benchmark for unreplicated experimental-design analysis

The purpose of this paper is to present a summary of the most extensive benchmark conducted to assess the performance of nine numerical techniques applied to analyze unreplicated experimental designs. These designs have been previously shown to be relevant to reliability growth programs. The numerical techniques evolved out of the difficulty in using the classical analysis of variance methods when the measured response was not replicated. Since they are of precious value under these circumstances, it was considered important to assess their statistical performance under typical experimental conditions. The authors introduce a figure of merit to rank the techniques according to their ability to identify active factors and reject spurious ones. Using this figure of merit they show that, in spite of their great conceptual differences, the nine techniques perform similarly.