The role of meta-learners in the adaptive selection of classifiers

Abstract

The use of machine learning techniques able to classify source code components in defective or not received a lot of attention by the research community in the last decades. Previous studies indicated that no machine learning classifier is capable of providing the best accuracy in any context, highlighting interesting complementarity among them. For these reasons ensemble methods, that combines several classifier models, have been proposed. Among these, it was proposed ASCI (Adaptive Selection of Classifiers in bug predIction), an adaptive method able to dynamically select among a set of machine learning classifiers the one that better predicts the bug proneness of a class based on its characteristics. In summary, ASCI experiments each classifier on the training set and then use a meta-learner (e.g., Random Forest) to select the most suitable classifier to use for each test set instance. In this work, we conduct an empirical investigation on 21 open source software systems with the aim of analyzing the performance of several classifiers used as meta-learner in combination with ASCI. The results show that the selection of the meta-learner has not strong influence in the results achieved by ASCI in the context of within-project bug prediction. Indeed, the use of lightweight classifiers such as Naive Bayes or Logistic Regression is suggested.