Project Features That Make Machine-Learning Based Fault Proneness Analysis Successful

Over the past years, we have witnessed the extensive use of various software fault proneness prediction techniques utilizing machine learning. These techniques use data from multiple sources representing various facets of the software systems being investigated. In spite of the complexity and performance of all such techniques and approaches proposed by the research community, we cannot yet expertly reason on the features which may render a software system a good or bad candidate for their application. In this paper, we build on the corpus of established machine learning approaches, and we perform an evaluation of system-wide process metrics versus the results acquired by the indiscriminate application of a published best set of classifiers. More specifically, we analyze the fault proneness prediction results obtained by applying a combination of the best classifiers and file features to 207 open source projects in order to identify which project features make a system suitable for Machine Learning based fault proneness analysis or not. Based on this analysis, we propose a meta-evaluator of the overall nature of a system that can be used to gauge in advance the performance that can be expected when applying the selected technique in terms of the key performance measures namely: Accuracy, Fl-measure, Precision, Recall and ROC-AUC.