An Empirical Analysis of Threshold Techniques for identifying Faulty Classes

Abstract

The experimental validation of the proficiency of the proposed techniques is a mandatory task in the research regime. The existing literature has proved the presence of extensive work attained the statistical validation of software metrics by utilizing them in the development of fault prediction models, where, both statistical and machine learning techniques were engaged into the construction of the models being capable of identifying faulty and non-faulty classes. On the contrary, the research area involving the investigation of threshold concept has not gained sufficient maturity. An effective threshold technique can assist in the identification of optimal cut-off value in software metric which can discriminate the faulty from non-faulty classes with minimal misclassification rate. The idea of threshold calculation can make the applicability of the existing metrics in software industries, a much easier task. As the developers only need to know the cut-off values which can help them to concentrate on the specific classes that exceeds the computed thresholds. Also, the presence of peculiarity in the software metric index can alert the testers and in turn helps them to disburse the resources systematically. The current study empirically validated and compared the discriminating strength of two threshold techniques, i.e., ROC curve and Alves Rankings, on the public dataset. This study selected twenty Object Oriented (OO) measures for the process of threshold calculation. Besides, the widely addressed metric suite proposed by Chidamber and Kemerer, this study also considered other fourteen OO measures for the experiment. Furthermore, Wilcoxon signed ranks test was used to enquire the classification difference between the aforementioned threshold techniques. The outcome from the statistical analysis revealed the better predictive capability of ROC curve than the Alves Rankings.