Understanding the nature of software evolution

Abstract

Over the past several years, we have been developing methods of measuring the change characteristics of evolving software systems. Not all changes to software systems are equal. Some changes to these systems are very small and have low impact on the system as a whole. Other changes are substantial and have a very large impact of the fault proneness of the complete system. In this study we will identify the sources of variation in the set of software metrics used to measure the system. We will then study the change characteristics to the system over a large number of builds. We have begun a new investigation in these areas in collaboration with a flight software technology development effort at the Jet Propulsion Laboratory (JPL) and have progressed in resolving the limitations of the earlier work in two distinct steps. First, we have developed a standard for the enumeration of faults. This new standard permits software faults to be measured precisely and accurately. Second, we have developed a practical framework for automating the measurement of these faults. This new standard and fault measurement process was then applied to a software system's structural evolution during its development. Every change to the software system was measured and every fault was identified and tracked to a specific code module. The measurement process was implemented in a network appliance, minimizing the impact of measurement activities on development efforts and enabling the comparison of measurements across multiple development efforts. In this paper, we analyze the measurements of structural evolution and fault counts obtained from the JPL flight software technology development effort. Our results indicate that the measures of structural attributes of the evolving software system are suitable for forming predictors of the number of faults inserted into software modules during their development, and that some types of change are more likely to result in the insertion of faults than others. The new fault standard also insures that the model so developed has greater predictive validity.