Evolutionary measures and their correlations with the performance of cross-version defect prediction for object-oriented projects

Abstract

Cross-version defect prediction (CVDP) for evolutionary projects has attracted much attention from researchers in recent years. For multiple versions of an object-oriented project, the degree of evolution (e.g., the degree of class change) between successive versions can reflect the differences between versions, which could affect the performance of CVDP. Therefore, how to measure the degree of evolution between successive versions and explore the correlations with the performance of CVDP are very important for software defect prediction. Based on the successive versions of evolutionary projects, this paper proposes six evolutionary measures from three aspects of class change, metric change, and label change, including the Ratio of New Classes (RNC), the Ratio of Deleted Classes (RDC), the Average Ratio of Metric Change (ARMC), the Ratio of Label Changed Classes (RLCC), the Ratio of Unchanged Classes (RUC), and the Ratio of Interference Classes (RIC). An empirical study was conducted on 40 versions of 11 object-oriented projects from the PROMISE repository. Precision, Recall, F-measure, and AUC were used as the performance indicators. Three correlation approaches (Pearson, Spearman, and Kendall) are applied to show the correlations between evolutionary measures and the performance of CVDP. The statistical results show that RNC, RDC, and RUC show no correlation with four performance indicators. ARMC shows weak or medium positive correlations with Recall and F-measure. RLCC and RIC show very strong or strong negative correlations with Recall and F-measure. The results indicate that the correlations between the proposed evolutionary measures and the performance of CVDP are different, which can guide the training set selection of CVDP.