Slope-based fault-proneness thresholds for software engineering measures

Abstract

Background. Practical use of a measure X for an internal attribute (e.g., size, structural complexity, cohesion, coupling) of a software module often requires setting a threshold on X, to make decisions as to which software modules may be estimated to be potentially faulty. To keep quality under control, practitioners may want to set a threshold on X to identify "early symptoms" of possible faultiness of a module, which should be closely monitored and possibly modified. Objective. We propose and evaluate an approach to setting a threshold on X to identify "early symptoms" of possible faultiness of software modules. Method. Our proposal is based on the existence of a statistically significant model that relates X to fault-proneness, defined as the probability that a module contains at least one fault. The curve representing a fault-proneness model is usually fairly "flat" for relatively small values of X and becomes steeper and steeper for larger values of X. We define two ways in which values of X can be used as "early symptoms" of possible faultiness. First, we use the value of X where the fault-proneness model curve changes direction the most, i.e., has maximum convexity. Second, we use the value in which the slope of the curve reaches a proportion (e.g., one half) of the maximum slope that is relevant for the developers. Results. First, we provide the theoretical underpinnings for our approach. Second, we show the empirical results obtained by applying our approach to data from the PROMISE repository by using fault-proneness models built via Binary Logistic and Probit regressions. Our results show that the proposed thresholds are actually effective in showing "early symptoms" of possible faultiness of a module, while achieving a level of accuracy in classifying faulty modules that is fairly close to other typical fault-proneness thresholds. Conclusions. Our method can be practically used for setting "early symptom" thresholds based on evidence captured by statistically significant models. In particular, the threshold based on the maximum convexity depends on characteristics of the models alone, so software project managers do not need to devise the thresholds themselves. If they choose to use the other kind of slope-based threshold, software project managers can choose a different proportion based on the level of risk-aversion they need when recognizing early symptoms of faultiness.