Defining metric thresholds for software product lines: a comparative study

A software product line (SPL) is a set of software systems that share a common and variable set of features. Software metrics provide basic means to quantify several modularity aspects of SPLs. However, the effectiveness of the SPL measurement process is directly dependent on the definition of reliable thresholds. If thresholds are not properly defined, it is difficult to actually know whether a given metric value indicates a potential problem in the feature implementation. There are several methods to derive thresholds for software metrics. However, there is little understanding about their appropriateness for the SPL context. This paper aims at comparing three methods to derive thresholds based on a benchmark of 33 SPLs. We assess to what extent these methods derive appropriate values for four metrics used in product-line engineering. These thresholds were used for guiding the identification of a typical anomaly found in features' implementation, named God Class. We also discuss the lessons learned on using such methods to derive thresholds for SPLs.