Evaluating Fuzzy Analogy on incomplete software projects data

Abstract

Missing Data (MD) is a widespread problem that can affect the ability to use data to construct effective software development effort prediction systems. This paper investigates the use of missing data (MD) techniques with Fuzzy Analogy. More specifically, this study analyze the predictive performance of this analogy-based technique when using toleration, deletion or k-nearest neighbors (KNN) imputation techniques using the Pred(0.25) accuracy criterion and thereafter compares the results with the findings when using the Standardized Accuracy (SA) measure. A total of 756 experiments were conducted involving seven data sets, three MD techniques (toleration, deletion and KNN imputation), three missingness mechanisms (MCAR: missing completely at random, MAR: missing at random, NIM: non-ignorable missing), and MD percentages from 10 percent to 90 percent. The results of accuracy measured in terms of Pred(0.25) confirm the findings of a study which used the SA measure. Moreover, we found that SA and Pred(0.25) measure different aspects of technique performance. Hence, SA is not sufficient to conclude about the technique accuracy and it should be used with other metrics, especially Pred(0.25).