An empirical investigation into the cost-effectiveness of test effort allocation strategies for finding faults

Abstract

In recent years, it has been shown that fault prediction models could effectively guide test effort allocation in finding faults if they have a high enough fault prediction accuracy (Norm(Popt) > 0.78). However, it is often difficult to achieve such a high fault prediction accuracy in practice. As a result, fault-prediction-model-guided allocation (FPA) methods may be not applicable in real development environments. To attack this problem, in this paper, we propose a new type of test effort allocation strategy: reliability-growth-model-guided allocation (RGA) method. For a given project release V, RGA attempts to predict the optimal test effort allocation for V by learning the fault distribution information from the previous releases. Based on three open-source projects, we empirically investigate the cost-effectiveness of three test effort allocation strategies for finding faults: RGA, FPA, and structural-complexity-guided allocation (SCA) method. The experimental results show that RGA shows a promising performance in finding faults when compared with SCA and FPA.