An ordinal-time reliability model applied to "Big-Bang" suite-based testing

Abstract

System testing is often performed by means of a comprehensive test suite that spans the functional requirements of a software product ("Big Bang" testing). This suite is then run repeatedly after each modification or fix until a satisfactory pass rate is achieved. Such testing does not lend itself to treatment with traditional reliability models which, in keeping with their origins in hardware, assume that each fault is fixed upon discovery and that the relevant temporal variable is calendar or execution time. In this paper, a model using ordinal time, measured by baselined development configurations, is presented. The use of this model in combination with an empirical mapping of test case failures to product code faults allows for rapid tracking of testing progress. The use of the model as an aid to project management in the testing phase of a project is illustrated.